-- Logs begin at Thu 2022-06-16 17:00:35 UTC, end at Thu 2022-06-16 18:31:37 UTC. -- Jun 16 17:45:51.800965 dv-cinder-837543-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 17:45:51.810550 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 17:45:51 2022] *** Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: nodename: dv-cinder-837543-4-iscsi Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: machine: x86_64 Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: clock source: unix Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: pcre jit disabled Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: detected number of CPU cores: 4 Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: current working directory: / Jun 16 17:45:51.812745 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: detected binary path: /usr/bin/uwsgi-core Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: your processes number limit is 47680 Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: your memory page size is 4096 bytes Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: detected max file descriptor number: 2048 Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: lock engine: pthread robust mutexes Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: thunder lock: enabled Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 17:45:51.813321 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 17:45:51.858365 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: Python main interpreter initialized at 0x55ad990189c0 Jun 16 17:45:51.858365 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: python threads support enabled Jun 16 17:45:51.858365 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: your server socket listen backlog is limited to 100 connections Jun 16 17:45:51.858365 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: your mercy for graceful operations on workers is 90 seconds Jun 16 17:45:51.858772 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 17:45:51.858849 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: *** Operational MODE: preforking *** Jun 16 17:45:51.859106 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: *** uWSGI is running in multiple interpreter mode *** Jun 16 17:45:51.859106 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: spawned uWSGI master process (pid: 80341) Jun 16 17:45:51.859327 dv-cinder-837543-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 17:45:51.860214 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: spawned uWSGI worker 1 (pid: 80353, cores: 1) Jun 16 17:45:51.860311 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: spawned uWSGI worker 2 (pid: 80354, cores: 1) Jun 16 17:45:51.861788 dv-cinder-837543-4-iscsi devstack@keystone.service[80341]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 17:45:58.357340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:45:58.357340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: from cryptography.utils import int_from_bytes Jun 16 17:45:58.358130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:45:58.358130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: from cryptography.utils import int_from_bytes Jun 16 17:45:58.407695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:45:58.407695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: from cryptography.utils import int_from_bytes Jun 16 17:45:58.408246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:45:58.408246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: from cryptography.utils import int_from_bytes Jun 16 17:45:58.408441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:45:58.408642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:45:58.408839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] command line args: [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:45:58.409122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:45:58.409305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ================================================================================ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:45:58.409569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] admin_token = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.409824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_dir = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.410016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.410202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_source = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.410386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] debug = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.410591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.410959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.411143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.411422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.411681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.411919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.412103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_config_append = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.412288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.412504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_dir = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.412638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.412882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.413135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.413387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.413635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.413843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.414645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.414904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.415159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.415426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.415666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.415905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.416142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.416396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.416647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.416932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.417424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.417608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] publish_errors = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.417846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.418080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.418261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.418509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.418747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.418987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] standard_threads = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.419222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.419498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.419681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.419861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_journal = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.420040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_json = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.420217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_stderr = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.420394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_syslog = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.420627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.420886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.421127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.421390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.421633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.421884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.422126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.422376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.422613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.422863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.423112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.423384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.423627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.423876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.424113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.424362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.424617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.424855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.425113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.425372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.425620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.425858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.426114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.426378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.426891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.427144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.427399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.427639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.427874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.428111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.428363 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.428614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.428849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.429083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.429332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.429608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.429850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.430102 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.430337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.430577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.430815 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.431048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.431280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.431544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.431777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.432014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.432265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.432498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.432734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.432967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.433204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.433440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.433690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.433924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.434161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.434397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.434636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.434873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.435109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.435341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.435612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.435849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.436087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.436320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.436583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.436824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.437056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.437292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.437530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.437799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.438041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.438288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.438534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.438770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.439005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.439242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.439516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.439759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.439997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.440241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.440478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.440722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.440950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.441186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.441421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.441686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.441928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.442172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.442410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.442652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.442887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.443123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.443359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.443612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.443842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.444072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.444308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.444542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.444783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.445019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.445294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.445537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.445790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.446053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.446298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.446532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.external = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.446765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.447066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.447302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.447560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.password = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.447793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.token = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.448035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.448271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.448518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.448751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.448999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.449246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.449477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.449712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.449960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.450207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.450456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.450697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.450941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.451176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.451464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.451722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.451960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.452211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.452497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.452742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.453006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.453248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.453506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.453743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.453979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.454227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.454480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.454714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.454961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.455201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.455482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.455723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.455982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.456221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.456468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.456704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.456953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.457038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:45:58.457182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:45:58.457270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.457375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] command line args: [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:45:58.457454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.457649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.457727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:45:58.457826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ================================================================================ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:45:58.457917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.458082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] admin_token = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.458161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.458330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_dir = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.458413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.458521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.458612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.458817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.458897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_source = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.459046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] debug = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.459125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.459257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.459388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.459521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.459611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.459686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.459765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.459937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.460018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.460183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.460263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.460416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.460511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.460587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_config_append = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.461238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.461331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.461534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.461813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.461920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_dir = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.462019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.462101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.462254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.462336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.462494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.462583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.462728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.462807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.462964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.463046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.463208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.463385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.463385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.463535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.463645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.463737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.463893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.463974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.464130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.464211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.464363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.464443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.464596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.464676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.464827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.464908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.465073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.465159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.465318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.465398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.465560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.465640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.465792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.466031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.466109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.466246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] publish_errors = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.466333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.466481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.466561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.466713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.466801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.466895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.466974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.467141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.467222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.467395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.467491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.467631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] standard_threads = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.467713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.467863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.467950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.468105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.468194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.468268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.468364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.468444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_journal = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.468601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.468680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_json = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.468760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_stderr = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.468838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.468933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_syslog = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.469060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.469148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:45:58.469293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.469404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.469536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.469626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.469798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.469879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.470852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.471965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.472921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.473964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.474908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.475869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.476014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.476096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.476265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.476408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.477127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.477540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.477701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.477779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.477853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.478974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.479981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.480926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.481945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.482874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.483981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.484920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.485915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.486886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.487999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.488919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.489926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.490878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.491842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.492140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.492479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.492674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.493060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.493306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.493551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.493803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.494036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.494280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.494581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.494847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.495964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.496872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.497987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.external = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.498937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.499026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.499173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.499254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.password = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.499327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:45:58.499504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.token = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.499746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.499982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.500227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.500458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.500705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.500951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.501180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.501412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.501656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.501902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.502149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.502385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.502627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.502860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.503121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.503388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.503632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.503886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.504174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.504419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.504681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.504922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.505175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.505408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.505647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.505890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.506137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.506367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.506612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.506850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.507100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.507335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.507668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.508572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.508842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.509085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.509339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.509591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.509829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.510064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.510313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.510546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.510783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.511031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.511286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.511543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.511776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.512007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.512253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.512498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.512745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.513001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.513249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.513521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.513756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.513991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.514219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.514454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.514682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.514916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.515162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.515423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.515676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.515906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.516154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.516402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.516633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.516876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.517119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.517347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.517583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.517810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.518044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.518276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.518522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.518753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.518986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.519236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.519506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.519739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.519968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.520214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.520461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.520699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.520932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.521166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.521394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.521638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.521900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.522130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.522373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.522617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.522860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.523093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.524039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.524844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.525094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.525338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.525574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.525829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.526078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.526331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.526577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.526822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.527054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.527301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.527561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.527796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.528030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.528262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.528505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.528750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.528983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.529234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.529463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.529705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.529936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.530168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.530400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.530644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.530883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.531111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.531346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.531609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.531859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.532108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.532344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.532593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.532844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.533075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.533319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.533553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.533788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.534020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.534263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.534491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.534738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.534982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.535227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.535495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.535742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.535992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.536236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.536483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.536713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.536957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.537200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.537445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.537688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.538130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.538365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.538612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.538997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.542359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.543141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.543429 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.543715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.543981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.544235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.544478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.544722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.544961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.545217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.545481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.545734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.545974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.546211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.546444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.546677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.546916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.547167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.547429 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.547692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.547944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.548188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.548440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.548684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.548932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.549169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.549407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.549640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.549888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.550139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.550371 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.550613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.550929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.551194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.551461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.551970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.552209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.552451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.552688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.552925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.553163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.553399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.553645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.553881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.554127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.554361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.555188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.556078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.556330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.556564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.556809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.557046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:45:58.557218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:45:58.616015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.616244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.616472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.626039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.655219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.655514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.670347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.670578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.670800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.679339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.699127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.699375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.710533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.738431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.770367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.770552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.776241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.776478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.839106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.839347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.839575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.839797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.840018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.840147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.840258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.840412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.840735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.841043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.841347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.841656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.864665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.864905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.865983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.866066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.866198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.866446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.878020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.878334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.891379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.891616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.891712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.891848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.891962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.892966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:45:58.910696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.910921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:45:58.911233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.911464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:45:58.911738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.912001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.912262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.912487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.912575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.912702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:45:58.913050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.913250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:45:58.913514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.914576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.914576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.914914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.924149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.924432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.924706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.924993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.925667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.925950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.926219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.926771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.934475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.935620 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.937810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.938096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.938382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.938664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.938907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.938998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.939187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.939280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.939485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.939570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.939763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.939844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.940841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.941110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.941378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.941642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.967761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.977254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.977445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.977445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.977709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.977709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.977995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.977995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.977995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.978355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.978355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.978812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.983991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.984283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.986634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.986923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.988865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.989154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.989441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.989730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.990011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.990293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.990569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.990841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.991858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:58.992178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.992458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.992740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.993016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.993285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.993551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:58.993813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.010093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.010381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.013081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.013364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.015052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.018263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.018744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.019168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.022689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.023764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.024834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.028561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.029259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.032375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.032687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.032896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.037300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.037919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.053904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.054194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.054549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.054958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.058097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.058387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.058654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.058916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.063048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.064824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.065114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.067056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.068899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.069122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.069813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.070209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.070699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.071925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.071925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.073911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.074197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.074484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.074769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.075053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.084938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.085223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.085520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.086437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.086847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.087114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.087457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.089238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.089528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.089806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.090086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.090365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.090737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.091014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.105979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.109284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.109710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.112642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.113003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.114379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.116086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.116327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.116414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.116650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.118201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.119879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.120165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.120446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.124261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.125908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.127704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.129180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.129328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.129466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.129866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.130156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.132622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.133166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.133426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.133705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.139470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.139762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.140044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.142748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.143042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.143321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.148004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.148831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.149108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.149706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.149706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.150042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.150321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.150598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.150878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.151149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.151448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.151722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.152008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.152102 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:45:59.152278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.152389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.152669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.152944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.153225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.153621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.153898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.154175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.154464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.154735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.155005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.155270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.155563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.155823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:45:59.191020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55ad990189c0 pid: 80354 (default app) Jun 16 17:45:59.194445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55ad990189c0 pid: 80353 (default app) Jun 16 17:45:59.194599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a487-29cd-4f80-b697-e09b8a8a83f9 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:45:59.194941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a487-29cd-4f80-b697-e09b8a8a83f9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:45:59.195399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8a487-29cd-4f80-b697-e09b8a8a83f9 None None] PATH_INFO: `/v3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:45:59.199641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Thu Jun 16 17:45:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:08.635659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:08.636162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:08.636162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:08.640474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:08.646038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:08.646419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:08.646600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:08.717659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: INFO dbcounter [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] Registered counter for database keystone Jun 16 17:46:08.739297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG oslo_db.sqlalchemy.engines [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80354) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:46:08.745817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writer thread running {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:46:08.933516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG passlib.handlers.bcrypt [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80354) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:46:08.934221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG passlib.handlers.bcrypt [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80354) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:46:08.992125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:08.993867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:09.128982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 17:46:09.291213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:09.362014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: INFO dbcounter [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] Registered counter for database keystone Jun 16 17:46:09.381777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG oslo_db.sqlalchemy.engines [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80353) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:46:09.388304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writer thread running {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:46:09.513289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b6416-a8a1-4e22-b9fe-7b57de26886b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:09.517597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:09.518279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:09.518567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:09.518847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:09.527967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:09.579607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING keystone.common.rbac_enforcer.enforcer [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:46:09.579948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478629d5-dc23-4df3-9b7d-9254d580c2e8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:09.590523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Jun 16 17:46:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:09.596536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:09.613967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef621b37-0d52-4105-bdbe-a79b70abb17f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:09.618910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:09.620065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:09.620349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:09.620626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] PATH_INFO: `/v3/projects/24af7229251445019ca42ea240bb437e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:09.622137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] RBAC: Authorizing `identity:get_project(project_id=24af7229251445019ca42ea240bb437e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:09.673138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:46:09.673684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d99b0-7665-4cf9-8a1d-cca33bc07a45 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:09.678408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Thu Jun 16 17:46:09 2022] GET /identity/v3/projects/24af7229251445019ca42ea240bb437e => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:11.871093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9df56-1f3e-421d-83c9-bd8789dd3e52 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.871647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9df56-1f3e-421d-83c9-bd8789dd3e52 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.871731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9df56-1f3e-421d-83c9-bd8789dd3e52 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.875837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:11.881308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a059-3502-407b-a21f-4f167887e804 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.882298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a059-3502-407b-a21f-4f167887e804 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.882420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958a059-3502-407b-a21f-4f167887e804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.901193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4958a059-3502-407b-a21f-4f167887e804 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:11.902731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4958a059-3502-407b-a21f-4f167887e804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:11.910673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 17:46:11.934760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9df56-1f3e-421d-83c9-bd8789dd3e52 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:11.947675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9df56-1f3e-421d-83c9-bd8789dd3e52 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:11.951936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c27a3-925f-40c5-87b8-be16b3911d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:11.952576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c27a3-925f-40c5-87b8-be16b3911d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.952892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c27a3-925f-40c5-87b8-be16b3911d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.953174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c27a3-925f-40c5-87b8-be16b3911d9c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.961046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c27a3-925f-40c5-87b8-be16b3911d9c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:11.962859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c27a3-925f-40c5-87b8-be16b3911d9c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:11.971866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Thu Jun 16 17:46:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:46:11.977473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958a059-3502-407b-a21f-4f167887e804 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:11.986772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958a059-3502-407b-a21f-4f167887e804 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:11.991835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73596e1-cb47-47fe-9244-bacd161399ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:11.992694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73596e1-cb47-47fe-9244-bacd161399ce admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:11.992979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73596e1-cb47-47fe-9244-bacd161399ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:11.993256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73596e1-cb47-47fe-9244-bacd161399ce admin admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:11.994674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73596e1-cb47-47fe-9244-bacd161399ce admin admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:11.996509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73596e1-cb47-47fe-9244-bacd161399ce admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:11.998072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Thu Jun 16 17:46:11 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:14.202122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ed8fb-9425-4332-9519-36864413f0ff None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.202742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ed8fb-9425-4332-9519-36864413f0ff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.202742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ed8fb-9425-4332-9519-36864413f0ff None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.204282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:14.209148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3f986-5482-4a90-bb31-b9745ca45503 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.210101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3f986-5482-4a90-bb31-b9745ca45503 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.210225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3f986-5482-4a90-bb31-b9745ca45503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.227117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b0a3f986-5482-4a90-bb31-b9745ca45503 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:14.228333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b0a3f986-5482-4a90-bb31-b9745ca45503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:14.312240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 6/11] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:14.336169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ed8fb-9425-4332-9519-36864413f0ff None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:14.456929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a5e5-860b-463f-9bc0-e187a40e51cc None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.457130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a5e5-860b-463f-9bc0-e187a40e51cc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.457415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a5e5-860b-463f-9bc0-e187a40e51cc None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.458797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 7/12] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:14.465023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.465311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.465600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.482440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:14.483702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:14.492714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ed8fb-9425-4332-9519-36864413f0ff None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:14.500637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:14.546580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:14.548001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.548503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.548992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.558609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 8/13] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:14.570334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:14.581401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:14.581909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:14.582842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:14.583876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 6/14] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:14.589786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:14.625201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:14.688173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c54c-c449-479b-b49e-2ba4cf3f6a41 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:14.697108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54abf68-c5fa-405c-8dcf-7ae75161f2a3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:14.723274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:14.723886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.724175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.724465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.732065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:14.732740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.733030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.733310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.734066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:14.734377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:14.735845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:14.746957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:14.748374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 7/15] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:14.754018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:14.794939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:14.796461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:14.850998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:14.856847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b5bc2-30de-4e13-8fa8-1f3e6d13de1c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:14.892656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:14.893535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:14.893824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:14.894107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:14.895773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:14.897291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:14.898582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 9/16] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:46:14.898965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 8/17] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:14 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:14.904149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:14.904273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:14.956954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:15.004840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec75e54-6d2f-45cc-9ae6-9bcc7c5a1e02 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:15.005117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a93eb9-5142-4b16-a6bc-495d4ac1f0cb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:15.042047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:15.042569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:15.042843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:15.043142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:15.043397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:15.043490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:15.043682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:15.043960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:15.044762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:15.046217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:15.047902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:15.055134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:15.057039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:15.058519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:46:15.080170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:15.081076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:15.081905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:46:15.081905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:46:15.081905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:46:15.081905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:15.081905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 10/19] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:15.087253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:15.091641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:15.167947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bb784-4159-41db-8a8f-bc8c09dbf304 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:15.197246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721d3d9-dff0-43b4-bedb-1284a655cec2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:15.199724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:15.208161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:15.208906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:15.209085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:15.211667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:15.211667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.domain.id=default, user.id=d088ec4f4ba04369a9931bed85bdd327) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:15.211667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:15.212119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44904c-53db-4dad-81ee-8e7a84b13b9d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:15.221069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Jun 16 17:46:15 2022] GET /identity/v3/role_assignments?user.id=d088ec4f4ba04369a9931bed85bdd327&scope.domain.id=default&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 233 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:15.239501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:15.240214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:15.240502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:15.241415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:15.242190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:15.242429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:15.243942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2a16c-6a41-4f0a-bb88-50d8b5ca68f0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:15.264103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 11/21] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:17.823794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3345-b227-4386-9dc0-1ca3906ff95a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:17.824836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3345-b227-4386-9dc0-1ca3906ff95a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:17.825322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a3345-b227-4386-9dc0-1ca3906ff95a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:17.826881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:17.832608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cac5a-8296-4a67-b9f0-7d932a6810b5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:17.833487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cac5a-8296-4a67-b9f0-7d932a6810b5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:17.834017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cac5a-8296-4a67-b9f0-7d932a6810b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:17.860902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0b7cac5a-8296-4a67-b9f0-7d932a6810b5 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:17.862493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b7cac5a-8296-4a67-b9f0-7d932a6810b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:17.954021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:17.989319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a3345-b227-4386-9dc0-1ca3906ff95a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:18.103462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a454-568b-488c-be44-afdeaf52d9d3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.104099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a454-568b-488c-be44-afdeaf52d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.104588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3a454-568b-488c-be44-afdeaf52d9d3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.106190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:18.112496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.113108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.113593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.154454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:18.160653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:18.162505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:18.176448 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a3345-b227-4386-9dc0-1ca3906ff95a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:18.237412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75d194-365f-4613-9afa-80ccf7b22131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:18.238602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75d194-365f-4613-9afa-80ccf7b22131 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.239131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75d194-365f-4613-9afa-80ccf7b22131 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.239662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75d194-365f-4613-9afa-80ccf7b22131 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.263271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75d194-365f-4613-9afa-80ccf7b22131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1f75d194-365f-4613-9afa-80ccf7b22131 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:18.274090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:18.274853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:18.277482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 12/25] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:18.279474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361558e-27f3-49b5-b116-ec68528e6661 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.280013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361558e-27f3-49b5-b116-ec68528e6661 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.280511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361558e-27f3-49b5-b116-ec68528e6661 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.282280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 13/26] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:18.288891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.289497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.289982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.307655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG passlib.handlers.bcrypt [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80353) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:46:18.308460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG passlib.handlers.bcrypt [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80353) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:46:18.320971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 14/27] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:18.321871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:18.357102 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:18.359191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:18.370453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:18.455993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 14/28] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:18.464563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:18.471161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b46ac-2cbb-4f22-bc02-fe13146e2c2d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:18.514300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:18.514984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.515270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.515578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.516580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:18.516891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:18.519147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c1605-f358-4960-a3e6-0aae35ec2fe0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:18.535078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 15/29] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:18.540586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8708dd-4b7b-4a21-9b90-d121f8adc965 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.542209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8708dd-4b7b-4a21-9b90-d121f8adc965 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.542500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8708dd-4b7b-4a21-9b90-d121f8adc965 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.543889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 16/30] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:18.544598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8708dd-4b7b-4a21-9b90-d121f8adc965 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:18.586152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:18.605549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef5376-497c-435e-b994-c47e4f90e1f2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:18.646012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:18.653998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:18.655601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.656213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.657030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.664112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8708dd-4b7b-4a21-9b90-d121f8adc965 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:18.682515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:18.684645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:18.701965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:18.702696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.702994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.703275 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.704058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:18.705528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:46:18.712615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:18.713411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:18.714249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:46:18.714249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:46:18.714249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:46:18.714249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:18.716281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 15/31] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:18.717336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:18.827491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eea1ea-9741-4e61-b829-7a1bdd860220 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:18.863253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198372b-e904-449c-8b7b-89f49be11bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:18.864365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198372b-e904-449c-8b7b-89f49be11bef None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.864788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198372b-e904-449c-8b7b-89f49be11bef None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.865226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198372b-e904-449c-8b7b-89f49be11bef None admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.867477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198372b-e904-449c-8b7b-89f49be11bef None admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:18.869220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198372b-e904-449c-8b7b-89f49be11bef None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:18.875985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 16/32] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:18 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:18.877736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:18.878214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:18.878628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:18.892852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:18.901423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:18.901423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:18.980898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 17/33] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:18.982128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:19.122364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4901c-8ef3-4635-b0cc-1bf1207e187b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:19.142833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 17/34] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jun 16 17:46:18 2022] POST /identity/v3/roles => generated 219 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:19.144513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:19.185127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:19.186224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.186957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.188343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.189338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:19.190093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:19.195879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:19.204979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:19.243246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 18/35] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:19.244507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:19.352950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9c6c4-ab0b-4b4d-bf7d-59ec02e3c21e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:19.407418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:19.407418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.407698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.407943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.414889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:19.417108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:19.418323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1bb44-ba3c-4686-a6ea-d25d2abff487 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:19.419204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 18/36] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:19.419961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:19.442606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:19.472803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13505f44-cbe1-4252-b704-bdfd3d717a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:19.473485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505f44-cbe1-4252-b704-bdfd3d717a24 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.474287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505f44-cbe1-4252-b704-bdfd3d717a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.474579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13505f44-cbe1-4252-b704-bdfd3d717a24 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.476101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13505f44-cbe1-4252-b704-bdfd3d717a24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:19.477552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13505f44-cbe1-4252-b704-bdfd3d717a24 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:19.540915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbea387-6c80-40f4-8181-5d9b1ce3ee2d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:19.580062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1bfea-d844-4756-989b-053d95b24dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:19.580736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1bfea-d844-4756-989b-053d95b24dc2 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.581024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1bfea-d844-4756-989b-053d95b24dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.581310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1bfea-d844-4756-989b-053d95b24dc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.582022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1bfea-d844-4756-989b-053d95b24dc2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:19.583486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1bfea-d844-4756-989b-053d95b24dc2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:19.682761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 19/37] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:46:19 2022] POST /identity/v3/projects => generated 301 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:19.684273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c63c7c-c484-4f5d-980c-57bca464d4a1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.684570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c63c7c-c484-4f5d-980c-57bca464d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.684853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c63c7c-c484-4f5d-980c-57bca464d4a1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.689878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 20/38] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:19.689878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c63c7c-c484-4f5d-980c-57bca464d4a1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:19.776904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:19.797737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 19/39] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jun 16 17:46:19 2022] POST /identity/v3/roles => generated 225 bytes in 553 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:19.799461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f598-1611-49a7-86cb-f7a8a0a03652 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.799759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f598-1611-49a7-86cb-f7a8a0a03652 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.800059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877f598-1611-49a7-86cb-f7a8a0a03652 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.801956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 20/40] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:19.807194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e1777-125e-4a83-b5ee-f3d3af07b828 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.807194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e1777-125e-4a83-b5ee-f3d3af07b828 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.807194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e1777-125e-4a83-b5ee-f3d3af07b828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.826056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e8e1777-125e-4a83-b5ee-f3d3af07b828 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:19.827768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e8e1777-125e-4a83-b5ee-f3d3af07b828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:19.833730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c63c7c-c484-4f5d-980c-57bca464d4a1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:19.879997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:19.881709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.882198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.882694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] PATH_INFO: `/v3/domains/default/users/d088ec4f4ba04369a9931bed85bdd327/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.886260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d088ec4f4ba04369a9931bed85bdd327, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:19.887800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:19.903018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:19.923351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 21/41] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:19.925106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:19.925404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:19.925690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:19.951520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:19.952798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:19.999148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 21/42] 10.222.0.58 () {58 vars in 1361 bytes} [Thu Jun 16 17:46:19 2022] PUT /identity/v3/domains/default/users/d088ec4f4ba04369a9931bed85bdd327/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:20.000932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:20.031612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 22/43] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:20.054733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:20.152302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081f662-a969-494a-8765-777e8fdeb8bc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:20.187940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:20.196037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15b356-a905-4cb5-a442-3730acc5e833 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:20.197358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:20.198238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.198539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.198827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.200175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:20.201654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:20.203500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 22/44] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:20.209036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:20.214538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:20.232935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca414e8-fd25-492d-b6ea-db1f63443409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:20.234106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca414e8-fd25-492d-b6ea-db1f63443409 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.234106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca414e8-fd25-492d-b6ea-db1f63443409 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.234356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca414e8-fd25-492d-b6ea-db1f63443409 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.235113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca414e8-fd25-492d-b6ea-db1f63443409 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:20.236823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca414e8-fd25-492d-b6ea-db1f63443409 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:20.312969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bacaab-40c5-45db-b8a0-46072a202939 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:20.343387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 23/45] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jun 16 17:46:20 2022] POST /identity/v3/roles => generated 223 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:20.368547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d04569-5ec0-43fc-8cd6-9269ed38e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:20.369350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d04569-5ec0-43fc-8cd6-9269ed38e627 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:20.370028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d04569-5ec0-43fc-8cd6-9269ed38e627 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:20.370028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d04569-5ec0-43fc-8cd6-9269ed38e627 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:20.370655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d04569-5ec0-43fc-8cd6-9269ed38e627 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:20.372214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d04569-5ec0-43fc-8cd6-9269ed38e627 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:20.494270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 23/46] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:46:20 2022] POST /identity/v3/projects => generated 312 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:22.200457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724efb7-0119-43de-a981-f10763f67dc1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.200989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724efb7-0119-43de-a981-f10763f67dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.201073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724efb7-0119-43de-a981-f10763f67dc1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.202439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 24/47] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:22.207715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.208124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.208284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.226136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:22.227702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:22.274968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:22.298654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 24/48] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:22.323565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724efb7-0119-43de-a981-f10763f67dc1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.434875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724efb7-0119-43de-a981-f10763f67dc1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:22.473170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:22.473170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.473455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.473537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.485927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:22.495168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:22.496121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:22.496874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 25/49] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:22.503556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.632625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16926ab1-a6db-44e8-9252-171d14de1cfa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:22.672578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:22.673273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.673557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.673837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.674605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:22.674911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:22.676610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae63e5-9f18-4aee-ba56-c2efaebe61e0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:22.686355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 25/50] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:22.692036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.718839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73ec58-8504-4767-9715-b0943f0c483c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.719138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73ec58-8504-4767-9715-b0943f0c483c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.719464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73ec58-8504-4767-9715-b0943f0c483c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.720841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 26/51] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:22.726355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.726586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.726876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.730653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:22.744684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:22.745892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:22.775966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:22.799652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321b0d7-83a2-4b24-a478-e7036803e2f6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:22.812730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 27/52] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:22.837370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.850813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:22.851731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.852024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.852313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.853552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:22.855031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:22.856748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 26/53] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:22 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:22.862396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.938353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abbe99-2930-4ab3-9cf5-f2b256743bba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:22.963283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493013e7-6b45-4273-a9e9-aeea3a7eb58f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:22.965615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:22.972232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:22.973038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.973317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.979819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:22.979819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:22.979819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:22.979819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 28/54] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:22.984316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:22.998010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:22.998893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:22.999130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:22.999446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:23.000812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:23.002464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:23.004747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 27/55] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:23.010543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:23.019715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:23.084062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22041a-76b7-4a67-8808-2dc9c48876b7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:23.109941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e38fc-6cc5-4bd9-a355-5625cd7e5b4f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:23.121518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8612b4-0aa5-4ce3-8ded-bc188d526e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:23.122194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8612b4-0aa5-4ce3-8ded-bc188d526e48 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:23.122483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8612b4-0aa5-4ce3-8ded-bc188d526e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:23.122769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8612b4-0aa5-4ce3-8ded-bc188d526e48 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:23.123545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8612b4-0aa5-4ce3-8ded-bc188d526e48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:23.124999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8612b4-0aa5-4ce3-8ded-bc188d526e48 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:23.147887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:23.148556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:23.148844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:23.149123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:23.149964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.domain.id=default, user.id=d088ec4f4ba04369a9931bed85bdd327) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:23.150275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:23.151741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2c359-1b13-43af-a8af-ae2b10fb0322 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:23.158833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 28/56] 10.222.0.58 () {56 vars in 1322 bytes} [Thu Jun 16 17:46:23 2022] GET /identity/v3/role_assignments?user.id=d088ec4f4ba04369a9931bed85bdd327&scope.domain.id=default&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:23.263271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 29/57] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Jun 16 17:46:22 2022] POST /identity/v3/projects => generated 298 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:25.523037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff73b19-647b-4944-b587-6350f6f66356 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:25.523581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff73b19-647b-4944-b587-6350f6f66356 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:25.523581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff73b19-647b-4944-b587-6350f6f66356 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:25.524922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 29/58] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:25.533078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:25.533554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:25.534582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:25.559945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:25.561153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:25.634197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 30/59] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:25.658636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff73b19-647b-4944-b587-6350f6f66356 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:25.695541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:25.764529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff73b19-647b-4944-b587-6350f6f66356 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:25.799533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f020c2-4638-4d9b-ac73-9b37d8f0b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:25.800247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f020c2-4638-4d9b-ac73-9b37d8f0b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:25.800529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f020c2-4638-4d9b-ac73-9b37d8f0b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:25.801064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f020c2-4638-4d9b-ac73-9b37d8f0b9f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:25.802244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f020c2-4638-4d9b-ac73-9b37d8f0b9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:25.804227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f020c2-4638-4d9b-ac73-9b37d8f0b9f9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:25.805851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 30/60] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:25.811511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:25.832109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:25.920731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871de98e-5d90-4ecf-aadd-7df481c72bf8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:25.956874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8ba0e-22c0-4261-a870-fef523d2dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:25.957560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8ba0e-22c0-4261-a870-fef523d2dfbe None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:25.957851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8ba0e-22c0-4261-a870-fef523d2dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:25.958136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8ba0e-22c0-4261-a870-fef523d2dfbe None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:25.959448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8ba0e-22c0-4261-a870-fef523d2dfbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:25.960937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8ba0e-22c0-4261-a870-fef523d2dfbe None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:26.086382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 31/61] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:25 2022] POST /identity/v3/users => generated 275 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:29.525780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c1ba-5175-41d5-b940-02d0711d04de None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.526822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c1ba-5175-41d5-b940-02d0711d04de None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.527328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4c1ba-5175-41d5-b940-02d0711d04de None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.529236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 31/62] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:29.534390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.535263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.535793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.560865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:29.562431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:29.650930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99f5c5-a79c-465d-9136-904e7a114c14 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.651658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99f5c5-a79c-465d-9136-904e7a114c14 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.652168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99f5c5-a79c-465d-9136-904e7a114c14 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.653762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 32/63] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:29.661196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c78ac-2d41-4fa1-afae-e555b2ccbeec None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.661888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c78ac-2d41-4fa1-afae-e555b2ccbeec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.662398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c78ac-2d41-4fa1-afae-e555b2ccbeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.685318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ae9c78ac-2d41-4fa1-afae-e555b2ccbeec None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:29.685900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 32/64] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:29.687355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ae9c78ac-2d41-4fa1-afae-e555b2ccbeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:29.713647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:29.718254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:29.845689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 33/65] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:29.847501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc7313-0dba-4c82-a21b-71405f013077 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.847946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc7313-0dba-4c82-a21b-71405f013077 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.848376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc7313-0dba-4c82-a21b-71405f013077 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.855401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 34/66] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:29.863146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484e97f-ef91-466d-9b6b-f2f7519525f2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.863683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484e97f-ef91-466d-9b6b-f2f7519525f2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.864107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484e97f-ef91-466d-9b6b-f2f7519525f2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.866336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 35/67] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:29.870148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced215c-d314-4c39-bc8f-a90b805b007f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:29.872431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.872895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.873328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.892138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:29.893403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:29.910665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a273da-3f45-4a80-8bc6-7ee42055980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:29.911981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a273da-3f45-4a80-8bc6-7ee42055980e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.912398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a273da-3f45-4a80-8bc6-7ee42055980e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.912879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a273da-3f45-4a80-8bc6-7ee42055980e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.924283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:29.925938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a273da-3f45-4a80-8bc6-7ee42055980e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a3a273da-3f45-4a80-8bc6-7ee42055980e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:29.936745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:29.945635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 33/68] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:29.946970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:29.948998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:29.948998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:29.963952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:29.965282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:29.980538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 36/69] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:29.981241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.049730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.071393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 34/70] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:30.071393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.118871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566acb9b-68f8-44c0-80e5-f94835406a2c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.158278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.159099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.159396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.159705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.176772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:30.184671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:30.185549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:30.186319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 37/71] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:30.186963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.199894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983b00d-8fea-45f8-846a-76737f93d208 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.222163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.257108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.258693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.259210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.259739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.261811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:30.262382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:30.264921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:30.279989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 35/72] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:30.281427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.312288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2c1b5-5a12-4812-8038-b6e74c3a0891 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.341956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.347930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.348743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.349609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.349975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.360505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:30.368283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:30.369292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 38/73] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:30.370413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.392294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb932f0-c7f0-4e7e-a0a3-1ecb3a34d0fa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.434738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f3695-8c89-4f29-ab03-ed6cee1b9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.435605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f3695-8c89-4f29-ab03-ed6cee1b9830 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.435893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f3695-8c89-4f29-ab03-ed6cee1b9830 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.436175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f3695-8c89-4f29-ab03-ed6cee1b9830 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.447980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f3695-8c89-4f29-ab03-ed6cee1b9830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1a5f3695-8c89-4f29-ab03-ed6cee1b9830 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:30.456147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:30.457101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 36/74] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:30.463668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c7e01-9180-4c8a-a0bc-1b71aa9b0cca None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.464366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c7e01-9180-4c8a-a0bc-1b71aa9b0cca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.464366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c7e01-9180-4c8a-a0bc-1b71aa9b0cca None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.464366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 37/75] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:30.464366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c7e01-9180-4c8a-a0bc-1b71aa9b0cca None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.469617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.485929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975117c-72b2-4878-9b7b-cc31d02d179f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.523923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.524599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.524893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.525174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.525934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:30.526506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:30.527643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:30.542254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 39/76] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:30.543125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.577444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c7e01-9180-4c8a-a0bc-1b71aa9b0cca None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.594138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.611763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.612651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.612935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.613213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.628306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:30.629704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:30.632603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 38/77] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:30.633380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.652573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc7de0-5890-4211-9126-5e71cb9e85e2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.686774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.687485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.687778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.688065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.688819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:30.689130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:30.690518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d801d-1c36-4215-ab01-7abd1f8399de None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:30.699376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 40/78] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:30.700625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.700927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.701213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.718055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:30.719277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:30.721307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.744929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e9fbf-060c-41fb-8adf-2a703310654e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.784585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.785262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.785542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.785826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.786589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:30.786963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:30.788455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:30.797618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 39/79] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:30.798472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.807435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 41/80] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:30.808425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.840157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:30.907614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99672a4-5a84-491c-af92-f09319428c32 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.915669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2371a4-e3f9-48bf-8d1c-1d0ca9683f00 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:30.940588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.941470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.941758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.942039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.944100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:30.945340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:30.947129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 40/81] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:30.947876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:30.954864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:30.955553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:30.955843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:30.956123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:30.964761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.025754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.027244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.047564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583a3b8-80fa-4bac-b8b9-0303662bee38 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.050039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.082837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.083792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.084269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.084563 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.085819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.087286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.091515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 41/82] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:31.092515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.142466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 42/83] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:31.143342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.182580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.200395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b775c-bf27-4f9c-95cb-40fb83026209 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.233658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.234714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.235029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.235309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.236679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.238032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.239769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 42/84] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:31.240552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.248937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b2f0-8577-4234-8308-811bde580c11 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.288510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.289300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.289417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.289585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.289861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.291031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.292512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.294112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 43/85] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:31.296027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.347656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699b0fe-dcdd-4331-b842-5844b87786bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.383852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.384550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.384826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.385148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.397670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab2120-bf69-4639-9926-5b0d71c505d3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.405824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.439798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.440428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.440660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.441002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.456772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.458475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.501471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.513707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.515176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.578441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 43/86] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:31.578441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.639801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 44/87] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:31.640670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.649800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.690378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bae799-b945-4dd0-96d2-6c07664ae0c9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.725415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.726287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.726569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.726844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.728089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.729547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.731517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 44/88] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:31.732285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.746767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.748610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330423b-727e-49c2-9882-e5aac9418434 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.782869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.783628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.783915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.784191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.838063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228f32-f1eb-436c-9900-51f0f9ab323a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:31.852342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.853802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.861970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:31.872380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:31.875650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:31.875650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:31.875650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:31.875650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:31.875971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:31.947831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 45/89] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:31 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:31.948778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:31.954072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 45/90] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:31.954796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:32.043741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:32.086764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448d6a6-c8a4-4455-8f19-e467be56aabd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.087549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab78c63-bbbc-4988-a9aa-25d7639dd8cd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.101034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:32.131088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff0178-688c-49b9-a116-490897553473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.132197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0178-688c-49b9-a116-490897553473 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.132486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0178-688c-49b9-a116-490897553473 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.133654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff0178-688c-49b9-a116-490897553473 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.134058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff0178-688c-49b9-a116-490897553473 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.134522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.135422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.135527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff0178-688c-49b9-a116-490897553473 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.135705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.135988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] PATH_INFO: `/v3/projects/0f683e959cdc49f3861180dc3af9eb5f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.137188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=0f683e959cdc49f3861180dc3af9eb5f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.137292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 46/91] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:32.137989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff0178-688c-49b9-a116-490897553473 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:32.138610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.140508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 46/92] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:31 2022] GET /identity/v3/projects/0f683e959cdc49f3861180dc3af9eb5f => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:32.141248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:32.289269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff0178-688c-49b9-a116-490897553473 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.303583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6be120-b876-4e36-ace0-98879ea15bb5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.329672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:32.336225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.337066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.345705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.345705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.345705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=b1b8dc4aab034096af0d6b9ede657d79) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:32.345705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.345705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.347971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 47/93] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:32 2022] GET /identity/v3/role_assignments?user.id=b1b8dc4aab034096af0d6b9ede657d79&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=4ef493d63de24283a7dff2172a9c856c => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:32.348905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:32.351779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.354184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.354683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.355138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.356841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.358706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.361308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 47/94] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:32 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:32.361965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:32.434057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:32.472700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f3922-5bfe-42e2-a9cc-0663da2bbbf3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.509844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e928f3e-88a3-4172-b012-7f5f27e878de None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.531022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.532082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.532568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.533947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.535301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=0f683e959cdc49f3861180dc3af9eb5f, user.id=b1b8dc4aab034096af0d6b9ede657d79) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:32.536085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.540002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.548975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 48/95] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:32 2022] GET /identity/v3/role_assignments?user.id=b1b8dc4aab034096af0d6b9ede657d79&scope.project.id=0f683e959cdc49f3861180dc3af9eb5f&role.id=4ef493d63de24283a7dff2172a9c856c => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:32.550050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.550474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:32.551195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.551588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.552493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.557607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=d088ec4f4ba04369a9931bed85bdd327) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:32.557607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.557607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23ef2f-3789-4714-b5f8-0cf0e868a84a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.571720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 48/96] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:32 2022] GET /identity/v3/role_assignments?user.id=d088ec4f4ba04369a9931bed85bdd327&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:32.636827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:32.708095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f1970-e559-4df4-8f37-f7378f4374d3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:32.763019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:32.763985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:32.764437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:32.764857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:32.765878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] RBAC: Adding query filter params (role.id=89845870f208412e84a72ff363bae771, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=b1b8dc4aab034096af0d6b9ede657d79) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:32.766341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:32.768034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98349cb1-3f89-44d2-9e29-2798d820d5d5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:32.780792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 49/97] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:32 2022] GET /identity/v3/role_assignments?user.id=b1b8dc4aab034096af0d6b9ede657d79&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=89845870f208412e84a72ff363bae771 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:34.752805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f4aad-1388-4189-8e56-c2a9554cd189 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:34.753315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f4aad-1388-4189-8e56-c2a9554cd189 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:34.753387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f4aad-1388-4189-8e56-c2a9554cd189 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:34.754974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 49/98] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:34.761133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:34.762036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:34.762542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:34.801839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:34.803534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:34.902142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 50/99] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:34.938995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f4aad-1388-4189-8e56-c2a9554cd189 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:35.088701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f4aad-1388-4189-8e56-c2a9554cd189 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:35.097570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:35.132137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:35.133302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.133833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.134304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.147047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:35.155310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:35.156286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:35.157037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:35.159753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 50/100] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:35.167017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:35.240848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:35.325779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f91c6-3f01-4160-9ff2-7f2f693e9961 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:35.370800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:35.371890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.372375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.372847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.373800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:35.374302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:35.375946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:35.391034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 51/101] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:35.397332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:35.542161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06174fac-fe4f-47bc-b414-88e1fb06d1d2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:35.584644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f6b1f-fcac-4621-8cd2-6826940d81a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:35.585908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f6b1f-fcac-4621-8cd2-6826940d81a1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.586424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f6b1f-fcac-4621-8cd2-6826940d81a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.586894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f6b1f-fcac-4621-8cd2-6826940d81a1 None admin] PATH_INFO: `/v3/users/537328f8d118412f83c899b48961ccd9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.606343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f6b1f-fcac-4621-8cd2-6826940d81a1 None admin] RBAC: Authorizing `identity:get_user(user_id=537328f8d118412f83c899b48961ccd9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:35.608196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f6b1f-fcac-4621-8cd2-6826940d81a1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:35.610145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 51/102] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:35 2022] GET /identity/v3/users/537328f8d118412f83c899b48961ccd9 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:35.617011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:35.658036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f8821-2d04-4113-ae28-39bbac914a58 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.658650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f8821-2d04-4113-ae28-39bbac914a58 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.659155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f8821-2d04-4113-ae28-39bbac914a58 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.660737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 52/103] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:35.666581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df40655-7bbd-43e8-bf28-2e256993e43e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.667150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df40655-7bbd-43e8-bf28-2e256993e43e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.667651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df40655-7bbd-43e8-bf28-2e256993e43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.704170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7df40655-7bbd-43e8-bf28-2e256993e43e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:35.705790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7df40655-7bbd-43e8-bf28-2e256993e43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:35.758703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:35.827518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e355a-4884-4931-bc58-2a7535b4cf6f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:35.836187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 53/104] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 17:46:35.837878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abc2b5-5975-4eb0-a944-1317b3e391ff None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.838404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abc2b5-5975-4eb0-a944-1317b3e391ff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.838890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abc2b5-5975-4eb0-a944-1317b3e391ff None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.841035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 54/105] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:35.846436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.847040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.847593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.864204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:35.890725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:35.892441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:35.916235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3aea34-bf67-4e8c-9315-7e20a6016ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:35.916951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aea34-bf67-4e8c-9315-7e20a6016ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:35.917248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aea34-bf67-4e8c-9315-7e20a6016ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:35.917537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aea34-bf67-4e8c-9315-7e20a6016ef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:35.997546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 55/106] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:35.998623 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.041652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3aea34-bf67-4e8c-9315-7e20a6016ef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:36.043201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3aea34-bf67-4e8c-9315-7e20a6016ef1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:36.108523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.127646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f74a-da03-4f48-a839-56907805ec63 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.160694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 52/107] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:36.162679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69609d-ac95-41e1-b141-5ed3201361b4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.162997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69609d-ac95-41e1-b141-5ed3201361b4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.163269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69609d-ac95-41e1-b141-5ed3201361b4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.167593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 53/108] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:36.167593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69609d-ac95-41e1-b141-5ed3201361b4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.175897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31697a-9551-472f-bf5b-cc0ca34824cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.178045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31697a-9551-472f-bf5b-cc0ca34824cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.178045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31697a-9551-472f-bf5b-cc0ca34824cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.178045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31697a-9551-472f-bf5b-cc0ca34824cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.189099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.191250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31697a-9551-472f-bf5b-cc0ca34824cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3e31697a-9551-472f-bf5b-cc0ca34824cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:36.200646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:36.201601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:36.202367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 56/109] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:36.203576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f723b30-0db0-4aed-9994-29dcae9af4da None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.204322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f723b30-0db0-4aed-9994-29dcae9af4da None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.204322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f723b30-0db0-4aed-9994-29dcae9af4da None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.205690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 57/110] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:36.210725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f723b30-0db0-4aed-9994-29dcae9af4da None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.307344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69609d-ac95-41e1-b141-5ed3201361b4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.333004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f723b30-0db0-4aed-9994-29dcae9af4da None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.346962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d603a-7c6d-431e-b6a3-406ad19cb6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.347891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d603a-7c6d-431e-b6a3-406ad19cb6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.348191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d603a-7c6d-431e-b6a3-406ad19cb6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.348518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d603a-7c6d-431e-b6a3-406ad19cb6a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.360768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d603a-7c6d-431e-b6a3-406ad19cb6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7f5d603a-7c6d-431e-b6a3-406ad19cb6a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:36.369412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:36.370359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:36.371161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:36.373078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.373981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.374266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.374543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.375907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:36.376646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 54/111] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:36.377685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:36.378061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27e706-02f8-4f67-a299-2b4dc6a7ca98 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.378366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27e706-02f8-4f67-a299-2b4dc6a7ca98 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.378655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27e706-02f8-4f67-a299-2b4dc6a7ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.380493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 58/112] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:36.387344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.399149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0b27e706-02f8-4f67-a299-2b4dc6a7ca98 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:36.400415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b27e706-02f8-4f67-a299-2b4dc6a7ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:36.406430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.472040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 55/113] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:36.475503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.475958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.476247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.489305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.496630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:36.497887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:36.518166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2949961-7380-43e5-bd45-4feb0ca7b43b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.555431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.556116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.556417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.556706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.557460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:36.557774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:36.559171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:36.568141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 59/114] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:36.568141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.573191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 56/115] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:36.573191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.650983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.677349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab760d5-2d03-4858-b5c2-fd54d1d8015a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.677553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cac19-706d-4080-9c8b-b1fef671c0eb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.711829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.712076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.712509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.712612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.712803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.712899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.713086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.713192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.713892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:36.713986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] RBAC: Adding query filter params (role.id=3ce88f0555894d2690078253590cdce7, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=537328f8d118412f83c899b48961ccd9) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:36.714208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:36.714312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:36.715637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:36.715747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da880157-957f-4da3-ac3c-31efd7de9946 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:36.718797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.721586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 57/116] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/role_assignments?user.id=537328f8d118412f83c899b48961ccd9&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=3ce88f0555894d2690078253590cdce7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:36.722907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2940-05e8-46ab-926b-81eb7f564c19 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.723152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2940-05e8-46ab-926b-81eb7f564c19 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.724186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2940-05e8-46ab-926b-81eb7f564c19 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.725328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 58/117] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:36.726063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b2940-05e8-46ab-926b-81eb7f564c19 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.734672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 60/118] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:36.736629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.865022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d9295-8a18-44f1-b006-e1b11d1cf3fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.867558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b2940-05e8-46ab-926b-81eb7f564c19 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:36.911990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551dd75-a89e-48b2-bbbe-54bb10e3fb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.913023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551dd75-a89e-48b2-bbbe-54bb10e3fb40 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.913294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551dd75-a89e-48b2-bbbe-54bb10e3fb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.913577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551dd75-a89e-48b2-bbbe-54bb10e3fb40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.914902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:36.916092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.916387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.917071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.918431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:36.920355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:36.923056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 61/119] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:36.923979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:36.927015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551dd75-a89e-48b2-bbbe-54bb10e3fb40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:36.931192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f551dd75-a89e-48b2-bbbe-54bb10e3fb40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:36.936790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:36.937919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:36.938687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 59/120] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:36.940012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:36.943714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:36.943714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:36.960330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:36.961615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:37.029166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.041859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 60/121] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:37.042682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.060533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754de29-632e-4bc3-bec2-356c95787925 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.108513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.109325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.109613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.109894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.111100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.112584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.115214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 62/122] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:37.116031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.158514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a4351-f75f-4315-893d-56b1d49b3997 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.194155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.207286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.218372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.218372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.218372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.218372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.218372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.218372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 61/123] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:37.218875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.254367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0203e3c-97d5-4907-b265-b73b8e55ad79 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.298542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.304686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.305691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.306273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.306401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.307716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.309205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.311018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 63/124] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:37.311855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.354441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f334db2-fd1c-4d67-8997-6394de78da55 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.399542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.400627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.400627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.400848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.401560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:37.401879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.403296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.414778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 62/125] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:37.415946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.432158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fbe26-0734-4aa4-a9ea-f94182234c38 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.469276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.470104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.470405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.470703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.481028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.485490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:37.494066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:37.495776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 64/126] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:37.496572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.537569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a49040-0f94-4f43-bd0f-58783acb9468 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.560823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.575765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.576483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.576774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.577062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.577790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.579297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.613390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983a302-e1c0-486c-9e5c-06c9acf77778 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.655492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.656488 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.656825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.657105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] PATH_INFO: `/v3/projects/0f683e959cdc49f3861180dc3af9eb5f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.658302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] RBAC: Authorizing `identity:get_project(project_id=0f683e959cdc49f3861180dc3af9eb5f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.659788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.662332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 65/127] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/projects/0f683e959cdc49f3861180dc3af9eb5f => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:37.662332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.676973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 63/128] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:46:37 2022] POST /identity/v3/projects => generated 302 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:37.687522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.747335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.807072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f5092-ffb9-4c34-a543-ae6914a3d311 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.840950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209c65-4e52-4a86-8af2-2b0c6e21182e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:37.851833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.852797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.853088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.853372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807/users/b1b8dc4aab034096af0d6b9ede657d79/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.855746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=91b1159015f54fa786a5b4615137d807, user_id=b1b8dc4aab034096af0d6b9ede657d79, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.857314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.890493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:37.891457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:37.891751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:37.892036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:37.893615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:37.894813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:37.897295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 64/129] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:37.898195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:37.948307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:37.958763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 66/130] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:37 2022] PUT /identity/v3/projects/91b1159015f54fa786a5b4615137d807/users/b1b8dc4aab034096af0d6b9ede657d79/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:37.969342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:38.083673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560e13b-a96d-4db0-a2b9-e57fa8fc431a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:38.152272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:38.152885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.153174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.153455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.154232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:38.154549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:38.156202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:38.167419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 65/131] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:38.172490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:38.220296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1851a-80b6-48d7-8a6f-225e9fac39d6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:38.276861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:38.312988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349b12-5408-4bda-8bff-d4149aabbce2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:38.314984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:38.320744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:38.321684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.321971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.322252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] PATH_INFO: `/v3/projects/0f683e959cdc49f3861180dc3af9eb5f/users/b1b8dc4aab034096af0d6b9ede657d79/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.324340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] RBAC: Authorizing `identity:create_grant(project_id=0f683e959cdc49f3861180dc3af9eb5f, user_id=b1b8dc4aab034096af0d6b9ede657d79, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:38.326110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:38.355928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:38.357189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.357745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.358254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.359819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:38.361579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:38.364574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 66/132] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:38 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:38.366317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:38.490015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b4832-6dfd-4c15-9f2b-bf16cbf441e5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:38.494289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 67/133] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:37 2022] PUT /identity/v3/projects/0f683e959cdc49f3861180dc3af9eb5f/users/b1b8dc4aab034096af0d6b9ede657d79/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:38.495200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:38.549783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:38.551021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.551345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.552167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.553750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:38.555288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:38.557260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 67/134] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:38 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:38.562722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:38.670627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:38.689757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eef61-4f18-45a2-9041-1a7bd706616b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:38.759031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14297c7-25dd-41b2-a658-4904b39cb06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:38.760855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14297c7-25dd-41b2-a658-4904b39cb06c None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.761359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14297c7-25dd-41b2-a658-4904b39cb06c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.762012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14297c7-25dd-41b2-a658-4904b39cb06c None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807/users/b1b8dc4aab034096af0d6b9ede657d79/roles/89845870f208412e84a72ff363bae771` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.764515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14297c7-25dd-41b2-a658-4904b39cb06c None admin] RBAC: Authorizing `identity:create_grant(project_id=91b1159015f54fa786a5b4615137d807, user_id=b1b8dc4aab034096af0d6b9ede657d79, role_id=89845870f208412e84a72ff363bae771)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:38.766166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14297c7-25dd-41b2-a658-4904b39cb06c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:38.805837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7561fb-04e7-4a37-84f8-43b4e5bd6609 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:38.845803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:38.846691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:38.846982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:38.847267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:38.849288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:38.850785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:38.852544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 68/135] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:38 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:38.858239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:38.866180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 68/136] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:38 2022] PUT /identity/v3/projects/91b1159015f54fa786a5b4615137d807/users/b1b8dc4aab034096af0d6b9ede657d79/roles/89845870f208412e84a72ff363bae771 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:38.998348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9749d-1281-4f75-b995-609e8c46d812 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:39.005884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:39.042470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c062d-e1ad-4c2c-86fb-28725bab5222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:39.043321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c062d-e1ad-4c2c-86fb-28725bab5222 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:39.043756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c062d-e1ad-4c2c-86fb-28725bab5222 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:39.044367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c062d-e1ad-4c2c-86fb-28725bab5222 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807/users/d088ec4f4ba04369a9931bed85bdd327/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:39.047104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c062d-e1ad-4c2c-86fb-28725bab5222 None admin] RBAC: Authorizing `identity:create_grant(project_id=91b1159015f54fa786a5b4615137d807, user_id=d088ec4f4ba04369a9931bed85bdd327, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:39.048660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c062d-e1ad-4c2c-86fb-28725bab5222 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:39.120670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 69/137] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:38 2022] PUT /identity/v3/projects/91b1159015f54fa786a5b4615137d807/users/d088ec4f4ba04369a9931bed85bdd327/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:46:39.204484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7eb2-21cc-4c0d-9be2-f825f9ba892e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:39.205477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7eb2-21cc-4c0d-9be2-f825f9ba892e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:39.205953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b7eb2-21cc-4c0d-9be2-f825f9ba892e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:39.208170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 69/138] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:39.214334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:39.214641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:39.214945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:39.240817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:39.244749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:39.324694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 70/139] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:39.378516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b7eb2-21cc-4c0d-9be2-f825f9ba892e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:39.443120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:39.505585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b7eb2-21cc-4c0d-9be2-f825f9ba892e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:39.541709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:39.542527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:39.542817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:39.543094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:39.555048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:39.563055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:39.564014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:39.564765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 70/140] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:39.571121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:39.702921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:39.709339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f939b0-d27c-4678-a306-648ec7ff82f8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:39.749870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:39.750652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:39.751010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:39.751420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:39.752301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:39.752678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:39.754129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:39.767074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 71/141] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:39.773098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:39.897021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0e1f2-7f23-436b-a69b-98835fcccacb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:39.908115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:39.935680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:39.936827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:39.937252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:39.937823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] PATH_INFO: `/v3/users/537328f8d118412f83c899b48961ccd9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:39.939208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=537328f8d118412f83c899b48961ccd9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:39.940819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:39.942864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 71/142] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:39 2022] GET /identity/v3/users/537328f8d118412f83c899b48961ccd9 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:39.952475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:40.092072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca9479-d2c0-4081-be0f-6a9604ec3a92 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:40.133414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8eb1d-1739-47c2-81a1-9d3d2c075cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:40.134454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8eb1d-1739-47c2-81a1-9d3d2c075cea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:40.134840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8eb1d-1739-47c2-81a1-9d3d2c075cea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:40.135244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8eb1d-1739-47c2-81a1-9d3d2c075cea None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:40.136581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8eb1d-1739-47c2-81a1-9d3d2c075cea None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:40.138124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8eb1d-1739-47c2-81a1-9d3d2c075cea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:40.139968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 72/143] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:39 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:40.147442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:40.289645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e183a-782d-46ae-9add-1e098c10f8c0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:40.330862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd59a20-1719-4838-976c-757533dc7851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:40.332060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd59a20-1719-4838-976c-757533dc7851 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:40.332557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd59a20-1719-4838-976c-757533dc7851 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:40.333533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd59a20-1719-4838-976c-757533dc7851 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807/users/537328f8d118412f83c899b48961ccd9/roles/3ce88f0555894d2690078253590cdce7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:40.340715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd59a20-1719-4838-976c-757533dc7851 None admin] RBAC: Authorizing `identity:create_grant(project_id=91b1159015f54fa786a5b4615137d807, user_id=537328f8d118412f83c899b48961ccd9, role_id=3ce88f0555894d2690078253590cdce7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:40.342398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd59a20-1719-4838-976c-757533dc7851 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:40.417854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 72/144] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:40 2022] PUT /identity/v3/projects/91b1159015f54fa786a5b4615137d807/users/537328f8d118412f83c899b48961ccd9/roles/3ce88f0555894d2690078253590cdce7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:40.823058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014ea41-4515-47b4-877f-52f21bb5b0da None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:40.823577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014ea41-4515-47b4-877f-52f21bb5b0da None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:40.823970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014ea41-4515-47b4-877f-52f21bb5b0da None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:40.825612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 73/145] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:40.828426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41c062-8f9c-45fd-93ae-9a7524252aea None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:40.828882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41c062-8f9c-45fd-93ae-9a7524252aea None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:40.829357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41c062-8f9c-45fd-93ae-9a7524252aea None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:40.830869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 73/146] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:40.831281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:40.831721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:40.832208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:40.836828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:40.837284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:40.837771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:40.862827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:40.866256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:40.872505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:40.874714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:40.905432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:40.949577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:41.000088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 74/147] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:41.029664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 74/148] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:41.045784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:41.071774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:41.225880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c173c71-792e-4db4-b5d7-404182c9c600 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:41.290998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:41.291847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:41.292130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:41.292400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:41.293580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:41.295035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:41.296685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 75/149] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:41.298526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4078e2f-50fe-435d-9ac7-fc747781dbd4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:41.303833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:41.355191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:41.373129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:41.373955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:41.374247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:41.374528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:41.393098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:41.409532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:41.410395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:41.411389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 75/150] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:41.417652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:41.467720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:41.523287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c51320-ba4b-4bae-9f1b-af7f36efb98f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:41.598937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:41.599735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:41.600028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:41.600307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:41.601032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:41.602470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:41.607748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849a89e-2c2f-4914-8b22-1926abc5b29a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:41.653911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:41.655037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:41.655553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:41.656051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:41.657003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:41.657555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:41.659137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:41.669297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 76/151] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:41.674402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:41.738715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 76/152] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:41 2022] POST /identity/v3/users => generated 283 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:41.791632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:41.805323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f5f93-facf-409b-ab5c-43d1ab6cbc45 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:41.848630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edbf0f-141f-4638-8bc8-39bd7b050a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:41.849916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edbf0f-141f-4638-8bc8-39bd7b050a7c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:41.850410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edbf0f-141f-4638-8bc8-39bd7b050a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:41.850987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edbf0f-141f-4638-8bc8-39bd7b050a7c None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:41.852501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edbf0f-141f-4638-8bc8-39bd7b050a7c None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:41.854135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edbf0f-141f-4638-8bc8-39bd7b050a7c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:41.855968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 77/153] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:41 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:41.864782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:41.997178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:42.016801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5c64b-6710-4b31-8fc3-04ae528d145b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:42.063556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:42.064163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.064448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.152967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:42.154476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:42.229808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e04a64-4c12-48bb-b952-4aba10360f23 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.230519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e04a64-4c12-48bb-b952-4aba10360f23 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.231066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e04a64-4c12-48bb-b952-4aba10360f23 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.232932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 78/154] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:42.238273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce713b11-f9c6-47e8-8a97-a36332b778c1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.238870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce713b11-f9c6-47e8-8a97-a36332b778c1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.239350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce713b11-f9c6-47e8-8a97-a36332b778c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.247185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:42.264916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 77/155] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:42.272279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:42.278956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ce713b11-f9c6-47e8-8a97-a36332b778c1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:42.282453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ce713b11-f9c6-47e8-8a97-a36332b778c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:42.368190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 79/156] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:42.371157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8ef74-ba92-425a-8ea9-85294829902b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.371670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8ef74-ba92-425a-8ea9-85294829902b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.372217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8ef74-ba92-425a-8ea9-85294829902b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.373732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 80/157] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:42.380317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ece1-6c21-4d9c-b862-f4852ae3e17c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.381160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ece1-6c21-4d9c-b862-f4852ae3e17c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.381663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118ece1-6c21-4d9c-b862-f4852ae3e17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.411837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c118ece1-6c21-4d9c-b862-f4852ae3e17c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:42.413125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c118ece1-6c21-4d9c-b862-f4852ae3e17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:42.421886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab6b3-4ad0-4f6c-87d8-84cd9ed52906 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:42.470540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:42.472016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.472519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.473981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.476255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:42.478160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:42.481130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 78/158] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:42 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:42.482229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:42.506060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 81/159] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:42.508205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa2fa7-8c5c-4cfb-9aad-6b94f700592f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.508512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa2fa7-8c5c-4cfb-9aad-6b94f700592f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.508795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa2fa7-8c5c-4cfb-9aad-6b94f700592f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.510099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 82/160] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:42.510812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa2fa7-8c5c-4cfb-9aad-6b94f700592f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:42.537010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:42.552232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:42.621334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c9bbf-ae87-48b6-ab84-cfc586f43093 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:42.658153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa2fa7-8c5c-4cfb-9aad-6b94f700592f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:42.678488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13663980-ad5b-433a-b65c-af32386dd5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:42.679558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663980-ad5b-433a-b65c-af32386dd5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.679985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663980-ad5b-433a-b65c-af32386dd5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.680401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13663980-ad5b-433a-b65c-af32386dd5b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.696974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13663980-ad5b-433a-b65c-af32386dd5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-13663980-ad5b-433a-b65c-af32386dd5b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:42.706905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:42.708541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:42.709436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:42.712811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 79/161] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:42.714289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.714779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.715269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.719421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:42.719421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:42.719421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:42.719421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:42.719749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=b1b8dc4aab034096af0d6b9ede657d79) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:42.719901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:42.721732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:42.730685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 83/162] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:42 2022] GET /identity/v3/role_assignments?user.id=b1b8dc4aab034096af0d6b9ede657d79&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=4ef493d63de24283a7dff2172a9c856c => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:42.731621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:42.741461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:42.742715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:42.886880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 80/163] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:42.887213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:42.891768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:42.938389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208937f-fd34-4837-81d2-5913354c1277 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:42.970593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:43.005604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.005956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.006247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.006531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.024748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:43.033295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:43.034187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:43.034976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 84/164] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:46:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:43.035767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.067704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac27ea9-f312-40a7-928d-112078045632 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.115150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.116390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.116829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.117254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.118841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:43.119474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:43.121159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:43.133470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 81/165] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:43.134602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.169927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fc07b-421c-44e0-8ded-93e78813b7aa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.211817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.212875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.213135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.213509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.224948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:43.229508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:43.237324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:43.238292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:43.239736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 85/166] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:43.241011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.274592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:43.278894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bb960-7840-4070-923d-95d9edf0441b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.320279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.321079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.321420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.321773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.322640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:43.323012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:43.325074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:43.336090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 82/167] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:43.336840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.432938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83779f5-2e59-4e86-b145-f2f39668fd61 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.508386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.509290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.509580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.509862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.511129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:43.513225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:43.514898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 86/168] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:43.515704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.596700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a680fe-4b19-4015-80f7-b9668117f44d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.627482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:43.632532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.633226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.633519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.633804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.634566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:43.635286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:43.648918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:43.648918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 83/169] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:43.648918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.678325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:43.686785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0e0d-7c0b-4a97-914f-e4c1c1081bbb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.760002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538f1c3-5e03-4060-83dd-99a0d5f76d31 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:43.775331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.776246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.776534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.776816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] PATH_INFO: `/v3/users/b1b8dc4aab034096af0d6b9ede657d79` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.778033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] RBAC: Authorizing `identity:get_user(user_id=b1b8dc4aab034096af0d6b9ede657d79)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:43.779519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:43.781375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 87/170] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/users/b1b8dc4aab034096af0d6b9ede657d79 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:43.781922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:43.796315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:43.796981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:43.797269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:43.797551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:43.886614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:43.888486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:43.916358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.008334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 84/171] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:44.009278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:44.024045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f30bbe-48e7-4b18-a23f-2ba93b816d9f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:44.104691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:44.106021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.106526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.106996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] PATH_INFO: `/v3/users/d088ec4f4ba04369a9931bed85bdd327` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.112186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] RBAC: Authorizing `identity:get_user(user_id=d088ec4f4ba04369a9931bed85bdd327)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:44.114551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:44.116906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 88/172] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:43 2022] GET /identity/v3/users/d088ec4f4ba04369a9931bed85bdd327 => generated 247 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:44.118005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:44.138072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd95cbf-e112-4014-b695-e599766e46d6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:44.138263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.187404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.199414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36aae6-4d5f-4ae0-8a72-526fe30b596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:44.203385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36aae6-4d5f-4ae0-8a72-526fe30b596e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.203385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36aae6-4d5f-4ae0-8a72-526fe30b596e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.203385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36aae6-4d5f-4ae0-8a72-526fe30b596e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.288073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36aae6-4d5f-4ae0-8a72-526fe30b596e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:44.290807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36aae6-4d5f-4ae0-8a72-526fe30b596e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:44.349735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71700e-2dd2-4398-b755-39e97fd34337 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:44.421853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c45141-b500-48cb-a53e-268c6e43a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:44.422720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c45141-b500-48cb-a53e-268c6e43a153 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.423006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c45141-b500-48cb-a53e-268c6e43a153 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.423284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c45141-b500-48cb-a53e-268c6e43a153 None admin] PATH_INFO: `/v3/projects/0f683e959cdc49f3861180dc3af9eb5f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.425710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c45141-b500-48cb-a53e-268c6e43a153 None admin] RBAC: Authorizing `identity:get_project(project_id=0f683e959cdc49f3861180dc3af9eb5f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:44.427760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c45141-b500-48cb-a53e-268c6e43a153 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:44.432859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 89/173] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:44 2022] GET /identity/v3/projects/0f683e959cdc49f3861180dc3af9eb5f => generated 312 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:44.433836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e3438-4b0a-4f65-8208-dea16ca59ac1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.434361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e3438-4b0a-4f65-8208-dea16ca59ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.434361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e3438-4b0a-4f65-8208-dea16ca59ac1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.445966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 90/174] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:44.446796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e3438-4b0a-4f65-8208-dea16ca59ac1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:44.462464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 85/175] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:44.465995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.466289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.466568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.496497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:44.497731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:44.535725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.575878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.626040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 86/176] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:44.626842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:44.645688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e3438-4b0a-4f65-8208-dea16ca59ac1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:44.688480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:44.689536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.690059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.690555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.759714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301fda3-9cdc-401a-b4f1-f0e293673541 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:44.795570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:44.796240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.796679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.796955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.797785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=0f683e959cdc49f3861180dc3af9eb5f, user.id=b1b8dc4aab034096af0d6b9ede657d79) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:44.798107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:44.799577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:44.806916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 87/177] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:44 2022] GET /identity/v3/role_assignments?user.id=b1b8dc4aab034096af0d6b9ede657d79&scope.project.id=0f683e959cdc49f3861180dc3af9eb5f&role.id=4ef493d63de24283a7dff2172a9c856c => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:44.807804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:44.824908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:44.826789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:44.828609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.917906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb473c6-f091-4c2f-89d5-bb107dd4393b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:44.955170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:44.956052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:44.956192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:44.956383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:44.956641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:44.958158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:44.959640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:44.961902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 88/178] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:44 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:44.962742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.027975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 91/179] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:45.028827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.136933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:45.159381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd18a4-2ac2-4888-b5cf-a7f0557212f1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:45.233159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:45.233967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.234250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.234523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.251867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700c91f-f45c-409f-be5a-364742126c25 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:45.254688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:45.262580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:45.263545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:45.264364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 89/180] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:45.265564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.302291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:45.303300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.303831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.304310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.305374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] RBAC: Adding query filter params (role.id=89845870f208412e84a72ff363bae771, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=b1b8dc4aab034096af0d6b9ede657d79) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:45.305900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:45.307661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f886-35de-4157-8e27-049ec1e7b7d2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:45.314674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 92/181] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/role_assignments?user.id=b1b8dc4aab034096af0d6b9ede657d79&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=89845870f208412e84a72ff363bae771 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:45.316360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad38a4-f8e4-438e-a6d3-6da6c23f0997 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.316939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad38a4-f8e4-438e-a6d3-6da6c23f0997 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.317420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad38a4-f8e4-438e-a6d3-6da6c23f0997 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.318948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 93/182] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:45.324573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad38a4-f8e4-438e-a6d3-6da6c23f0997 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.366513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:45.380957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30951548-96f0-486d-90d2-376f83e15c99 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:45.421981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2760863-a6ce-470f-95cb-fce761dbc5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:45.422686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2760863-a6ce-470f-95cb-fce761dbc5bd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.422973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2760863-a6ce-470f-95cb-fce761dbc5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.423250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2760863-a6ce-470f-95cb-fce761dbc5bd None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.425747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2760863-a6ce-470f-95cb-fce761dbc5bd None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:45.427735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2760863-a6ce-470f-95cb-fce761dbc5bd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:45.429699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 90/183] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:45.430712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.430997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.431274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.462813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:45.464206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:45.481952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:45.532170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad38a4-f8e4-438e-a6d3-6da6c23f0997 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:45.549494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 91/184] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:45.551878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.619666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:45.620360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.620648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.620933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.621696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:45.622007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:45.624449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:45.639744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 94/185] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:45.640576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.716754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661b33-4531-4001-bfe2-8d5721a471a1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:45.762726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:45.766567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:45.767260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd82936-b072-484c-aad5-27857ab5afe1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:45.767658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.768134 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.768579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.771910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=d088ec4f4ba04369a9931bed85bdd327) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:45.771910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:45.773844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:45.786556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 92/186] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/role_assignments?user.id=d088ec4f4ba04369a9931bed85bdd327&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:45.788169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.810347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:45.836199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:45.837029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:45.837318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:45.837599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:45.861896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:45.884266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:45.885246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:45.886111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 95/187] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:45.892194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:45.989719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e139-b332-4e37-9c89-20553f7ae298 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.027687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dce74e-50fc-4ac0-9f6f-d6b491fcf665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.028564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce74e-50fc-4ac0-9f6f-d6b491fcf665 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.028847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce74e-50fc-4ac0-9f6f-d6b491fcf665 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.029123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dce74e-50fc-4ac0-9f6f-d6b491fcf665 None admin] PATH_INFO: `/v3/users/537328f8d118412f83c899b48961ccd9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.029382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad857be1-4df5-41d5-bfc6-8762c15ffb0b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.031316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dce74e-50fc-4ac0-9f6f-d6b491fcf665 None admin] RBAC: Authorizing `identity:get_user(user_id=537328f8d118412f83c899b48961ccd9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.033299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dce74e-50fc-4ac0-9f6f-d6b491fcf665 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.034546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 93/188] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/users/537328f8d118412f83c899b48961ccd9 => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:46.037977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32a518-e932-4cea-87e2-e0faa690681b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.038190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32a518-e932-4cea-87e2-e0faa690681b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.038748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32a518-e932-4cea-87e2-e0faa690681b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.039945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 94/189] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:46.041067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563fd6-4c8c-4521-9198-7ab6bf550cd3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.041290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563fd6-4c8c-4521-9198-7ab6bf550cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.041510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563fd6-4c8c-4521-9198-7ab6bf550cd3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.042884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 95/190] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:46.044287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9563fd6-4c8c-4521-9198-7ab6bf550cd3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.074966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.075699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.076063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.076341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.077091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:46.077400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.078893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375bbc7-74f6-4ed9-b771-b6b0b37f6276 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.087630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 96/191] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:46.088901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6d10-4749-4609-836e-05a1937a4c1f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.089187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6d10-4749-4609-836e-05a1937a4c1f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.089464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6d10-4749-4609-836e-05a1937a4c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.105456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bcbb6d10-4749-4609-836e-05a1937a4c1f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:46.106649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bcbb6d10-4749-4609-836e-05a1937a4c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:46.121063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.135190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.157791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9563fd6-4c8c-4521-9198-7ab6bf550cd3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.172059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 97/192] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:46.173336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.173631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.173910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.195077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:46.196456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:46.197823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.198601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.198771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.199045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.267570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.267740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 98/193] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:46.268638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.269782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.358129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.359118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.377811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c54d5e-13b4-4483-b77f-f86d76585c68 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.381253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 96/194] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:46.382096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.412478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.413373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.414019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.414452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] PATH_INFO: `/v3/users/f6d6b926e39747e292cfe881d57aa9df` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.428455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=f6d6b926e39747e292cfe881d57aa9df)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.430003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.431749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 99/195] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/users/f6d6b926e39747e292cfe881d57aa9df => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:46.432510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.481979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480eca5-1c9f-4d94-978a-e0afb4eaa4ca None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.515483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.516218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.516502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.516775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.526798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:46.531104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad272-634c-4af8-9fe4-eb5eea708ac0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:46.534908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:46.541307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 97/196] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:46.542195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.573181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.573977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.574268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.574553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.575770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.577219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.579742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 100/197] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:46.579863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.579955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.588811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.662262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082ea74-940f-4b76-a6a1-5a34507db1ed None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.688342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dbd3b-a1fb-4d01-a1a8-ff740eb98cc6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.697335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.698202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.698481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.698751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.699960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.701400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.703082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 98/198] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:46.703866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.728315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.728964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.729241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.729526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.796438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.797833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.801831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c30fb4-482c-49e9-bb98-86e390eb7354 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.804105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.805863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:46.835741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.836416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.836704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.836986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.837735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:46.838047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.839438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.847297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 99/199] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:46.848635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.898612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 101/200] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:46.899497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:46.951015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbd6f2-825e-4289-826f-3703c053bd1a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:46.985878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:46.986548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:46.986835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:46.987112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:46.987852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:46.989961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:46.997300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c16b1f-aad4-4bc1-aa53-98ff92bbee0f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.005793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:47.031941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.032586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.032872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.033147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.033965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] RBAC: Adding query filter params (role.id=3ce88f0555894d2690078253590cdce7, scope.project.id=91b1159015f54fa786a5b4615137d807, user.id=537328f8d118412f83c899b48961ccd9) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:47.034272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.036094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.041653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 102/201] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:46 2022] GET /identity/v3/role_assignments?user.id=537328f8d118412f83c899b48961ccd9&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=3ce88f0555894d2690078253590cdce7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:47.042412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.141831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 100/202] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:46 2022] POST /identity/v3/users => generated 297 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:47.152574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.158503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe4e0-d626-4ed5-96f8-852466a2c4dc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.195754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:47.203169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.204178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.204462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.204828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] PATH_INFO: `/v3/users/f6d6b926e39747e292cfe881d57aa9df` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.206245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] RBAC: Authorizing `identity:get_user(user_id=f6d6b926e39747e292cfe881d57aa9df)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.208167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.209943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 103/203] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/users/f6d6b926e39747e292cfe881d57aa9df => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:47.217161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.280317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:47.326943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261da8a-aa73-4397-a7c9-4a7e3539b360 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.368590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e272-5280-4492-9e8c-e8689f6faab2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.369581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.370462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.371548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.371548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.372563 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.374677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.375908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 101/204] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:47.382192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.407884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.408464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.408748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.409025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.498181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:47.499762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.501247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.508520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb29a8-0074-4640-9e98-0c7584d0539d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.551628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:47.556924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.557905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.558329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.558678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.559962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=f6d6b926e39747e292cfe881d57aa9df) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:47.560358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.562101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.573098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 102/205] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/role_assignments?user.id=f6d6b926e39747e292cfe881d57aa9df&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:47.638054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 104/206] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:47.640109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.766672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9888beb-0b8b-4f44-82e4-58f82a4a4280 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.806043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a8000-d64c-40c5-a4a9-63fa7d181001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.806917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a8000-d64c-40c5-a4a9-63fa7d181001 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.807201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a8000-d64c-40c5-a4a9-63fa7d181001 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.807518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a8000-d64c-40c5-a4a9-63fa7d181001 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.808691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a8000-d64c-40c5-a4a9-63fa7d181001 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.810123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a8000-d64c-40c5-a4a9-63fa7d181001 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.812141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 103/207] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:47.818593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:47.924744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed32d0-9f09-4568-a635-e6b696731587 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:47.960494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:47.961517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:47.962000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:47.962474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:47.963519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] RBAC: Adding query filter params (role.id=89845870f208412e84a72ff363bae771, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=f6d6b926e39747e292cfe881d57aa9df) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:47.964050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:47.965917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a7e9c-fc5d-417e-80f8-6158b47c254b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:47.972345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 105/208] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:47 2022] GET /identity/v3/role_assignments?user.id=f6d6b926e39747e292cfe881d57aa9df&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=89845870f208412e84a72ff363bae771 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:49.549037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b8ca1-36d0-47bc-b94d-28d1ada52ba7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.549525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0ba5-6536-4cb4-91eb-5b1a2020df5d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.549883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b8ca1-36d0-47bc-b94d-28d1ada52ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.549957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0ba5-6536-4cb4-91eb-5b1a2020df5d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.550276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b8ca1-36d0-47bc-b94d-28d1ada52ba7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.550370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0ba5-6536-4cb4-91eb-5b1a2020df5d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.552004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 104/209] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:49.552512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 106/210] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:49.557794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.558084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.558368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.560214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.560503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.560783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.567748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:49.581082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:49.584253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:49.584355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:49.584355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:49.602089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:49.659157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 105/212] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:49.659484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 107/212] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:49.679007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:49.684153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:49.791773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e38fae-3686-4576-865f-5587bd9b07ee None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:49.793412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfed398-a21d-4082-8779-07edb7267106 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:49.826075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c9b59-bf77-4c56-9c25-caf4dc71fceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:49.826308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:49.826794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c9b59-bf77-4c56-9c25-caf4dc71fceb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.827085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c9b59-bf77-4c56-9c25-caf4dc71fceb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.827381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c9b59-bf77-4c56-9c25-caf4dc71fceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.828575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c9b59-bf77-4c56-9c25-caf4dc71fceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:49.830023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c9b59-bf77-4c56-9c25-caf4dc71fceb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:49.832270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 106/213] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:49.832472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbd8aa-ec18-4305-b502-3b4f9dae4f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:49.833486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bc253-7b00-49f0-988f-4578fdf560c3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.833777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bc253-7b00-49f0-988f-4578fdf560c3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.833976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbd8aa-ec18-4305-b502-3b4f9dae4f33 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.834072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bc253-7b00-49f0-988f-4578fdf560c3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.834540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbd8aa-ec18-4305-b502-3b4f9dae4f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.834975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbd8aa-ec18-4305-b502-3b4f9dae4f33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.835455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 107/214] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:49.837682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bc253-7b00-49f0-988f-4578fdf560c3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:49.849475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbd8aa-ec18-4305-b502-3b4f9dae4f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-abbbd8aa-ec18-4305-b502-3b4f9dae4f33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:49.858461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:49.859341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:49.860273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:49.861311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:49.863197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 108/215] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:49.864528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:49.864971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:49.865253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:49.883185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:49.884423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:49.952422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bc253-7b00-49f0-988f-4578fdf560c3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:49.966463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 109/216] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:49.967148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.006084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.006770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.007297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.007528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.008188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:50.009648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:50.080105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280df56f-c01d-47e3-bfe0-2faaf9bda9af None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.104927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.118174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.118875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.119162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.119499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.120259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:50.120573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:50.121996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:50.132681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 110/217] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:50.133549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.142997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 108/218] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:49 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:46:50.148442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.249663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.275295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6a8cb-6325-4065-adeb-84efbe67fe93 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.281135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9103263-caa0-401a-8bfc-324ee96cc6e7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.319187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48c8e7-68be-415e-a5e5-e9c42c461f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.320883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c8e7-68be-415e-a5e5-e9c42c461f27 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.320883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c8e7-68be-415e-a5e5-e9c42c461f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.320883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c8e7-68be-415e-a5e5-e9c42c461f27 None admin] PATH_INFO: `/v3/users/4454b5da82ad43bca0694b902113418a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.322412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b2eec-c39b-4713-a069-f93a3b0179cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.323214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b2eec-c39b-4713-a069-f93a3b0179cf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.324315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b2eec-c39b-4713-a069-f93a3b0179cf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.324315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b2eec-c39b-4713-a069-f93a3b0179cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.337973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c8e7-68be-415e-a5e5-e9c42c461f27 None admin] RBAC: Authorizing `identity:get_user(user_id=4454b5da82ad43bca0694b902113418a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:50.339481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c8e7-68be-415e-a5e5-e9c42c461f27 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:50.341356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b2eec-c39b-4713-a069-f93a3b0179cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:50.341471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 109/219] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/users/4454b5da82ad43bca0694b902113418a => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:50.342423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2af1-6d6d-4a4e-bd51-7fcb1c939bf5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.343604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2af1-6d6d-4a4e-bd51-7fcb1c939bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.343604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2af1-6d6d-4a4e-bd51-7fcb1c939bf5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.344336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 110/220] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:50.346341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd2af1-6d6d-4a4e-bd51-7fcb1c939bf5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1e0b2eec-c39b-4713-a069-f93a3b0179cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:50.352629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:50.353550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:50.354351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 111/221] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:50.355139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.359657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.359657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.376301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:50.377496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:50.428046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.464477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 112/222] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:50.465358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.479846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd2af1-6d6d-4a4e-bd51-7fcb1c939bf5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.537102 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.537783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.538066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.538922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.552215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.591455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e064c0-07c4-4345-85ca-f456e8e41e2c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.624957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:50.626434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:50.633086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.633774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.634058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.634335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.635090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:50.635432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:50.637458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:50.650986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 113/223] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:50.652156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.701677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.752368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 111/224] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:46:50.753492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.775145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e887c-d704-4a3d-a0f3-8efdca3ef527 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.798263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.821136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.822390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.823307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.824152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.839740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:50.855529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:50.856437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:50.857225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:50.860037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 114/225] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:50.861200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.884961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb009ab-aca3-475d-b873-0bd92d48bfee None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:50.933934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:50.935108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:50.935670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:50.935957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] PATH_INFO: `/v3/users/f6d6b926e39747e292cfe881d57aa9df` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:50.937850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] RBAC: Authorizing `identity:get_user(user_id=f6d6b926e39747e292cfe881d57aa9df)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:50.939298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:50.941859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 112/226] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/users/f6d6b926e39747e292cfe881d57aa9df => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:50.943130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:50.976541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:50.982355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d0a6e-9e44-41c3-b9f8-5f2d805c8aff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.017599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.018467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.018752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.019026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.020972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.022193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.024036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 115/227] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:51.025073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.079390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd738f-1581-47f6-9aed-b2a016550b19 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.081267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:51.119500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.120317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.120720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.121101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.122145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:51.122557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.124089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.137688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 113/228] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:46:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:51.137688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.174421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980aeac-7c23-4c3c-b171-356d264b21f3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.226050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.227169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.227617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.228024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.229349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.230955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.233173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 116/229] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:51 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:51.234084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.275635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424712-8005-48ad-a03d-4e98ca25e74d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.281554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:51.313770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.314664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.315357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.315544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.316799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=4454b5da82ad43bca0694b902113418a) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:51.317320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.319273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.325808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 114/230] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:51 2022] GET /identity/v3/role_assignments?user.id=4454b5da82ad43bca0694b902113418a&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=4ef493d63de24283a7dff2172a9c856c => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:51.326881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.360680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:51.379064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc918854-d1b0-43a3-b99f-347d48e3ec15 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.425933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.427504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.427948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.428403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] PATH_INFO: `/v3/users/f6d6b926e39747e292cfe881d57aa9df` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.429910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] RBAC: Authorizing `identity:get_user(user_id=f6d6b926e39747e292cfe881d57aa9df)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.432393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.434724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 117/231] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:51 2022] GET /identity/v3/users/f6d6b926e39747e292cfe881d57aa9df => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:51.441023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.450149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35dca8-6b20-40f6-a4bb-c69aaa497479 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.489728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.490659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.491045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.491454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/f6d6b926e39747e292cfe881d57aa9df/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.493793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=816e8e0d63834df6ab20ef551cf3d9a1, user_id=f6d6b926e39747e292cfe881d57aa9df, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.495823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.575152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 115/232] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:51 2022] PUT /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/f6d6b926e39747e292cfe881d57aa9df/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:51.585288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:51.588762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093917e3-a9e2-4956-8dca-b845dab9b0a8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.634484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852388e-a3db-4d14-95b2-376efd5f8da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.635344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852388e-a3db-4d14-95b2-376efd5f8da8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.635657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852388e-a3db-4d14-95b2-376efd5f8da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.635935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852388e-a3db-4d14-95b2-376efd5f8da8 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.638102 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852388e-a3db-4d14-95b2-376efd5f8da8 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.639655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852388e-a3db-4d14-95b2-376efd5f8da8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.641500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 118/233] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:51 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:51.648451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:51.788089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:51.805306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35929e18-8bbb-4f18-adc3-5ba814f645b1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:51.851013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a9c34-3b5d-4f46-a56f-3edb84c89aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:51.851961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9c34-3b5d-4f46-a56f-3edb84c89aea None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:51.852249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9c34-3b5d-4f46-a56f-3edb84c89aea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:51.852520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a9c34-3b5d-4f46-a56f-3edb84c89aea None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/f6d6b926e39747e292cfe881d57aa9df/roles/89845870f208412e84a72ff363bae771` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:51.854762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a9c34-3b5d-4f46-a56f-3edb84c89aea None admin] RBAC: Authorizing `identity:create_grant(project_id=816e8e0d63834df6ab20ef551cf3d9a1, user_id=f6d6b926e39747e292cfe881d57aa9df, role_id=89845870f208412e84a72ff363bae771)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:51.856253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a9c34-3b5d-4f46-a56f-3edb84c89aea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:51.965323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 116/234] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:51 2022] PUT /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/f6d6b926e39747e292cfe881d57aa9df/roles/89845870f208412e84a72ff363bae771 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:52.672231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6868f3-28ee-42f6-b230-4d64e1911cb9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:52.673186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6868f3-28ee-42f6-b230-4d64e1911cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:52.673695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6868f3-28ee-42f6-b230-4d64e1911cb9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:52.676364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 119/235] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:52.684753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd52035-c4b0-42a4-b543-d76db72c06d1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:52.685605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd52035-c4b0-42a4-b543-d76db72c06d1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:52.686087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd52035-c4b0-42a4-b543-d76db72c06d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:52.712371 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fbd52035-c4b0-42a4-b543-d76db72c06d1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:52.713916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fbd52035-c4b0-42a4-b543-d76db72c06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:52.801458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 117/236] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:52.836666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6868f3-28ee-42f6-b230-4d64e1911cb9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:52.849889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6922-ec0f-4d82-9023-b42b4bdd7ab2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:52.849889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6922-ec0f-4d82-9023-b42b4bdd7ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:52.850147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6922-ec0f-4d82-9023-b42b4bdd7ab2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:52.851657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 118/237] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:46:52.856555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:52.856845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:52.857126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:52.883456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:52.884666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:52.983409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 119/238] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:53.025162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6868f3-28ee-42f6-b230-4d64e1911cb9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.066577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:53.078429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.081570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.081570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.082264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.082756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.105090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:53.118578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:53.119522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:53.120773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 120/239] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:53.127336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:53.191950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.247062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7b086-de36-45ff-adaf-9447bd6cb1ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.297156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.297963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.298347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.298634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.300405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:53.301858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:53.303509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 120/240] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:53.313980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:53.344870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd3de5-17f4-455c-8ee7-364f8e4b7afc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.412524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.413197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.413485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.413766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.414519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:53.414831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:53.416234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:53.435334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 121/241] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:53.442735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:53.468500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.549627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e65b89-126b-4aad-a6bb-41fbfbc58ceb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.562179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.573810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84886f5f-47dd-49d2-9609-10d6215cf9ad None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.630431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.631691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.632199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.632932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] PATH_INFO: `/v3/users/6a59a8a18b1a4ee985cfa7060b16b29b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.645271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.646248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.646714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.647195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.648126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:53.649851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:53.663173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] RBAC: Authorizing `identity:get_user(user_id=6a59a8a18b1a4ee985cfa7060b16b29b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:53.664930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:53.668309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 122/242] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:53 2022] GET /identity/v3/users/6a59a8a18b1a4ee985cfa7060b16b29b => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:53.677833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:53.758429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 121/243] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:53 2022] POST /identity/v3/users => generated 293 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:53.784150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284ef1f-0f2a-4389-8ad8-f35cd1c6fb2f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:53.823309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b26061-cc1f-4936-af84-69a41a256d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:53.823971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b26061-cc1f-4936-af84-69a41a256d16 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:53.824255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b26061-cc1f-4936-af84-69a41a256d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:53.824569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b26061-cc1f-4936-af84-69a41a256d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:53.845095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:53.916468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b26061-cc1f-4936-af84-69a41a256d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:53.917961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b26061-cc1f-4936-af84-69a41a256d16 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:54.046946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 123/244] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:54.053384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:54.147359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a1fb4-ba04-4e3a-b5ac-516b9abcb405 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.148266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a1fb4-ba04-4e3a-b5ac-516b9abcb405 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.148815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a1fb4-ba04-4e3a-b5ac-516b9abcb405 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.150685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 124/245] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:54.158684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.159242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.160338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.179190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:54.179870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:54.203983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b554050-f119-4b57-8bcb-0438cad15dcd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:54.240756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:54.249203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:54.254889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:54.255822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.256112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.257194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.257653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:54.259278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:54.262518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 122/246] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:54 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:54.268497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:54.286926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 125/247] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:54.316512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:54.416711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ad3e3-e6ec-4879-a70d-7bc7917a70c3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:54.449528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70038889-a26a-4082-83b7-29c0319f5ce3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:54.454759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:54.455485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.455778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.456062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.457449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] RBAC: Adding query filter params (role.id=3ce88f0555894d2690078253590cdce7, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=6a59a8a18b1a4ee985cfa7060b16b29b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:54.457769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:54.459220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43176e-6991-4569-91c9-f8896c9aa9d8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:54.466689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 123/248] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:54 2022] GET /identity/v3/role_assignments?user.id=6a59a8a18b1a4ee985cfa7060b16b29b&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=3ce88f0555894d2690078253590cdce7 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:54.468649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8ae0f-fa04-4860-8464-066328958bb4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.468941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8ae0f-fa04-4860-8464-066328958bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.469219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8ae0f-fa04-4860-8464-066328958bb4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.470725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 124/249] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:54.477604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.477892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.478177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.511080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:54.512368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:54.520190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:54.521017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.521302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.521581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.543979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:54.553788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:54.554756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:46:54.555585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:46:54.560995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 126/250] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:54.568502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:54.647612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:54.658973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:54.667217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 125/251] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:54.698438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:54.767467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb899fe-c1a3-447a-9a92-967d7b4d4efc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:54.811392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:54.812276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.812563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.812841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.813920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:54.814236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:54.816222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:54.830822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 127/252] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:54.837214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:54.912958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e5175-9a4d-4dea-b8a5-db03ab119dd7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:54.972012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc73994-92c2-413c-a73b-33c10c8a3bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:54.972966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc73994-92c2-413c-a73b-33c10c8a3bab None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:54.973253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc73994-92c2-413c-a73b-33c10c8a3bab None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:54.973533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc73994-92c2-413c-a73b-33c10c8a3bab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:54.997517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc73994-92c2-413c-a73b-33c10c8a3bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dfc73994-92c2-413c-a73b-33c10c8a3bab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:55.006049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:55.006968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:55.007804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:55.017670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 126/253] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:55.022931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9338-1314-4408-9ef3-32ffbae47cdf None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.023219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9338-1314-4408-9ef3-32ffbae47cdf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.023525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9338-1314-4408-9ef3-32ffbae47cdf None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.030387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 127/254] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:55.034218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9338-1314-4408-9ef3-32ffbae47cdf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:55.035542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:55.040641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7ab25-f2ed-4e0c-8662-283ecebfd312 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:55.109924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:55.110776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619f71-eb8e-4978-bdb2-ec98fb27360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:55.111698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619f71-eb8e-4978-bdb2-ec98fb27360c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.111984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619f71-eb8e-4978-bdb2-ec98fb27360c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.112268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619f71-eb8e-4978-bdb2-ec98fb27360c None admin] PATH_INFO: `/v3/users/f6d6b926e39747e292cfe881d57aa9df` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.120155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619f71-eb8e-4978-bdb2-ec98fb27360c None admin] RBAC: Authorizing `identity:get_user(user_id=f6d6b926e39747e292cfe881d57aa9df)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:55.121638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619f71-eb8e-4978-bdb2-ec98fb27360c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:55.128910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 128/255] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:54 2022] GET /identity/v3/users/f6d6b926e39747e292cfe881d57aa9df => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:55.128910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.128910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.128910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.153762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:55.156067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:55.190376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9338-1314-4408-9ef3-32ffbae47cdf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:55.264356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:55.265044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.265332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.265613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.266379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:55.267578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:55.269010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:55.280644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 129/256] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:55.281506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:55.288447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 128/257] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:55.294775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:55.454141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:55.459050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cb01-413b-4ed0-9893-ddc7aed07af2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:55.494733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca12d91-8bee-4a8c-b9f3-a89c0e5c5cd5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:55.515626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:55.525880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:55.529611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.529611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.529611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.546225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:55.547472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.547948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.548405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] PATH_INFO: `/v3/users/4454b5da82ad43bca0694b902113418a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.552702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] RBAC: Authorizing `identity:get_user(user_id=4454b5da82ad43bca0694b902113418a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:55.554404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:55.558737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 129/258] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:55 2022] GET /identity/v3/users/4454b5da82ad43bca0694b902113418a => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:55.559754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:55.679152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:55.681395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:55.783951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cf140-6280-441f-b0b2-f36905eaa3c0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:55.827540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:55.834908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 130/259] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:55.839815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:55.858092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:55.858918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:55.859203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:55.859511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:55.870861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:55.879412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:55.880524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:46:55.881597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:55.884190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 130/260] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:46:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:55.885366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:55.938855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:55.994269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda4346-4e22-4340-8130-35770a6e8f8a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.031214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.032500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.033013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.033493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.035064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.036841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:56.039656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 131/261] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:55 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:56.044986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:56.071299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270760fb-5bbe-4666-b24d-b9850383ccdd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.140064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.140940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.141220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.141492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.142707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.145022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:56.147078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 131/262] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:55 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:56.147881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:56.151743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c3908-f7c2-4071-9563-cb1e4a608693 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.175842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:56.198457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.199150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.199462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.199748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.200668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:56.200987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.202365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:56.217136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 132/263] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:46:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:56.220141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:56.287257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:56.334573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df757c91-cdd0-4432-96b0-2bd35c4a3b4a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.373067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.373734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.374029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.374311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.375460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=f6d6b926e39747e292cfe881d57aa9df) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:56.375786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.378729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:56.385740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 132/264] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:56 2022] GET /identity/v3/role_assignments?user.id=f6d6b926e39747e292cfe881d57aa9df&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:56.393017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:56.451394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d909f-7dd4-41d0-8f9f-c5562bbe05b4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.527777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.528742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.529035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.529315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/4454b5da82ad43bca0694b902113418a/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.533809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] RBAC: Authorizing `identity:create_grant(project_id=816e8e0d63834df6ab20ef551cf3d9a1, user_id=4454b5da82ad43bca0694b902113418a, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.535319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:56.588222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff6d7-2f9b-40df-84ae-6973efc5ea9a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.623599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 133/265] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:56 2022] PUT /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/4454b5da82ad43bca0694b902113418a/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:56.643471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:56.652049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32728a23-34f7-4e0e-a951-3f2a9a1813e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.652973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728a23-34f7-4e0e-a951-3f2a9a1813e8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.653264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728a23-34f7-4e0e-a951-3f2a9a1813e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.653546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728a23-34f7-4e0e-a951-3f2a9a1813e8 None admin] PATH_INFO: `/v3/users/f6d6b926e39747e292cfe881d57aa9df` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.656254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728a23-34f7-4e0e-a951-3f2a9a1813e8 None admin] RBAC: Authorizing `identity:get_user(user_id=f6d6b926e39747e292cfe881d57aa9df)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.657825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728a23-34f7-4e0e-a951-3f2a9a1813e8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:56.660961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 133/266] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:56 2022] GET /identity/v3/users/f6d6b926e39747e292cfe881d57aa9df => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:56.666715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:56.726705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:56.777767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0468-3f9c-4edd-ac5b-c4178a82b105 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.778130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0468-3f9c-4edd-ac5b-c4178a82b105 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.778468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0468-3f9c-4edd-ac5b-c4178a82b105 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.782050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 134/267] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:56.789463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.790069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.790359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.823609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:56.824848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:56.834690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55a2ea-d3d4-40f2-a6c1-9f7726a12bec None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:56.893129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68797b-e5e8-4f9e-aeaf-5a008565471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:56.894218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68797b-e5e8-4f9e-aeaf-5a008565471f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:56.894707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68797b-e5e8-4f9e-aeaf-5a008565471f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:56.895204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68797b-e5e8-4f9e-aeaf-5a008565471f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:56.938167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 135/268] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:46:56.965826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:56.986413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68797b-e5e8-4f9e-aeaf-5a008565471f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:56.989317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68797b-e5e8-4f9e-aeaf-5a008565471f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.025957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.092068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c451ae-e0db-4ec0-9953-e81d18b95940 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.116300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 134/269] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:46:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:46:57.118095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f56cf-2b1a-453c-8741-29dbbf0f36f0 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.118586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f56cf-2b1a-453c-8741-29dbbf0f36f0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.119061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f56cf-2b1a-453c-8741-29dbbf0f36f0 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.120840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 135/270] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:57.121768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f56cf-2b1a-453c-8741-29dbbf0f36f0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.136200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.149792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cac21-e720-4c87-9e3b-5f9366012b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.149792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cac21-e720-4c87-9e3b-5f9366012b2b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.149792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cac21-e720-4c87-9e3b-5f9366012b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.150121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cac21-e720-4c87-9e3b-5f9366012b2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.151570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cac21-e720-4c87-9e3b-5f9366012b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:57.153036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cac21-e720-4c87-9e3b-5f9366012b2b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.156230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 136/271] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:46:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:57.161477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.161477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.161477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.181834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:57.183103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:57.272604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 137/272] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:57.275951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.282397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f56cf-2b1a-453c-8741-29dbbf0f36f0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.336068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.337243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.337667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.338081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.339625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:57.341415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.343633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 136/273] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:57.344536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.348590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.411051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d689dc-fa90-4fa2-8442-7340e8cfa1c4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.441272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.451201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.451907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.452187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.452639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.453196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:57.454681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.455450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4800e-3717-4991-a00c-d75cc6210b7a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.490899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85287a36-313f-4e6c-af7e-c5f4a36a6671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.491724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85287a36-313f-4e6c-af7e-c5f4a36a6671 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.492007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85287a36-313f-4e6c-af7e-c5f4a36a6671 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.492278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85287a36-313f-4e6c-af7e-c5f4a36a6671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.502689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85287a36-313f-4e6c-af7e-c5f4a36a6671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-85287a36-313f-4e6c-af7e-c5f4a36a6671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:57.510952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:57.512036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:57.512868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 137/274] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:57.513671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bc4e8-509a-425f-bf9d-6d4a1311536c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.513954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bc4e8-509a-425f-bf9d-6d4a1311536c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.514229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bc4e8-509a-425f-bf9d-6d4a1311536c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.520691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 138/275] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:57.520691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bc4e8-509a-425f-bf9d-6d4a1311536c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.558504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 138/276] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:46:57 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:46:57.564451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.594022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.644343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bc4e8-509a-425f-bf9d-6d4a1311536c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.691206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.692530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.693044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.693555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.694756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] RBAC: Adding query filter params (role.id=89845870f208412e84a72ff363bae771, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=f6d6b926e39747e292cfe881d57aa9df) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:57.695286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:57.697196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cbec7-9392-49fb-b9d9-8189f46b2005 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.710663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d691d0-0836-46c7-996e-ba3c795946f8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.711559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 139/277] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/role_assignments?user.id=f6d6b926e39747e292cfe881d57aa9df&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=89845870f208412e84a72ff363bae771 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:57.713297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.713911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.714447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.740874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:57.742095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:57.767036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.768121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.768667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.769850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.770871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:57.771265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:57.773841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.784278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 139/278] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:57.792449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.826661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 140/279] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:57.848228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.851641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:57.911941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85532e4c-045e-4481-8cc5-59b713477321 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.921049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:57.951333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:57.952895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:57.953350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:57.953809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] PATH_INFO: `/v3/users/e79e9ad9fa434b00a3d1340ff07a76d0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:57.970954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] RBAC: Authorizing `identity:get_user(user_id=e79e9ad9fa434b00a3d1340ff07a76d0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:57.973512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:57.976030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 140/280] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/users/e79e9ad9fa434b00a3d1340ff07a76d0 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:46:57.981143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b471-1ccd-4392-b76e-5735cf532cb5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:57.982345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:58.032170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:58.032991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:58.033277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:58.033553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:58.053170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:58.062954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:58.063930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:58.064726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 141/281] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:46:58.072798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:58.148558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46c560-ed7d-463e-aa38-bd3f3de259ec None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:58.150156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:58.191955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:58.193793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:58.194084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:58.194361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:58.195199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, user.id=e79e9ad9fa434b00a3d1340ff07a76d0, scope.system=all) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:58.195551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:58.197937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:58.204830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff6b5-0e70-4b2b-9598-b15a893249c8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:58.205298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 141/282] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jun 16 17:46:57 2022] GET /identity/v3/role_assignments?user.id=e79e9ad9fa434b00a3d1340ff07a76d0&scope.system=all&role.id=4ef493d63de24283a7dff2172a9c856c => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:58.228868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:58.255589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:58.256537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:58.256970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:58.257391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:58.258288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:58.258751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:58.260350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:58.271210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 142/283] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:58.279391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:58.451715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf92850-7d34-46bc-aa1c-e2c0690d6015 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:58.498116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:58.498915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:58.499198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:58.499501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] PATH_INFO: `/v3/users/6a59a8a18b1a4ee985cfa7060b16b29b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:58.500795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] RBAC: Authorizing `identity:get_user(user_id=6a59a8a18b1a4ee985cfa7060b16b29b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:58.502238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:58.505509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 142/284] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:58 2022] GET /identity/v3/users/6a59a8a18b1a4ee985cfa7060b16b29b => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:58.510583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:58.669368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6028196-cbe7-447b-a4a4-943daebd2314 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:58.729412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c91205-2c2d-4f34-a0f6-92e4762a7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:58.730592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91205-2c2d-4f34-a0f6-92e4762a7c4c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:58.731080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91205-2c2d-4f34-a0f6-92e4762a7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:58.731551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91205-2c2d-4f34-a0f6-92e4762a7c4c None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:58.732934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c91205-2c2d-4f34-a0f6-92e4762a7c4c None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:58.734733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c91205-2c2d-4f34-a0f6-92e4762a7c4c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:58.736665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 143/285] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:46:58 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:58.743059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:58.786757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:58.876859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbebf8d-7fb5-4c76-8e2e-127a9b8a33dd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:58.933224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209cb32-2987-4e6d-b436-3cd7671a5018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:58.934118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209cb32-2987-4e6d-b436-3cd7671a5018 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:58.934402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209cb32-2987-4e6d-b436-3cd7671a5018 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:58.934676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209cb32-2987-4e6d-b436-3cd7671a5018 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/6a59a8a18b1a4ee985cfa7060b16b29b/roles/3ce88f0555894d2690078253590cdce7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:58.940143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209cb32-2987-4e6d-b436-3cd7671a5018 None admin] RBAC: Authorizing `identity:create_grant(project_id=816e8e0d63834df6ab20ef551cf3d9a1, user_id=6a59a8a18b1a4ee985cfa7060b16b29b, role_id=3ce88f0555894d2690078253590cdce7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:58.941650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209cb32-2987-4e6d-b436-3cd7671a5018 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:59.263446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 143/286] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:46:58 2022] PUT /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/users/6a59a8a18b1a4ee985cfa7060b16b29b/roles/3ce88f0555894d2690078253590cdce7 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:46:59.317230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9be67-f2a1-4b62-aab5-aae4dd317517 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:59.317785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9be67-f2a1-4b62-aab5-aae4dd317517 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:59.317785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9be67-f2a1-4b62-aab5-aae4dd317517 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:59.319635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 144/287] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:46:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:46:59.327755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:59.327755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:59.327755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:59.349508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:46:59.350773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:46:59.460818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 144/288] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:46:59.490003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9be67-f2a1-4b62-aab5-aae4dd317517 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:59.548610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:59.608122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9be67-f2a1-4b62-aab5-aae4dd317517 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:59.654894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:59.656090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:59.656619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:59.657125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:59.670797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:46:59.680131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:46:59.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:46:59.681946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:46:59.684875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 145/289] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:46:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:46:59.690985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:46:59.734886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:46:59.834626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0741ab-078e-4347-a39f-62710c2351da None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:46:59.886417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:46:59.887100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:46:59.887455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:46:59.887692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:46:59.888444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:46:59.888753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:46:59.890153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:46:59.903248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 145/290] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:46:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:46:59.908467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:00.049041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8674da-104d-4dd1-a301-6ed87313ee2f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:00.057136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:00.091082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce4c97-9af4-45d9-86f6-cd4ca2d75052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:00.092349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4c97-9af4-45d9-86f6-cd4ca2d75052 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.092854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4c97-9af4-45d9-86f6-cd4ca2d75052 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.093344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce4c97-9af4-45d9-86f6-cd4ca2d75052 None admin] PATH_INFO: `/v3/users/4454b5da82ad43bca0694b902113418a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.094807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce4c97-9af4-45d9-86f6-cd4ca2d75052 None admin] RBAC: Authorizing `identity:get_user(user_id=4454b5da82ad43bca0694b902113418a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:00.096544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce4c97-9af4-45d9-86f6-cd4ca2d75052 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:00.099893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 146/291] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:46:59 2022] GET /identity/v3/users/4454b5da82ad43bca0694b902113418a => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:00.105470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:00.248409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:00.271936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024f6e6-f38e-42c4-b9e0-9d77047830bd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:00.309573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8e47-b79f-4ac7-8d59-7e78cea62c45 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.310220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8e47-b79f-4ac7-8d59-7e78cea62c45 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.310765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8e47-b79f-4ac7-8d59-7e78cea62c45 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.313536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 147/292] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:00.320401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.321403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.321844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:00.322155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.322493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.322776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.323048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.356377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:00.357493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:00.438494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:00.440005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:00.486031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 148/293] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:00.508361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:00.583435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:00.615173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 146/294] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:00.621303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:00.657567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd8647-f945-4af6-90dc-a1443116361e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:00.660644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:00.712486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:00.713869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.717027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.717532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.718959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:00.721866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:00.723979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 149/295] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:00.731682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:00.837769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e554942-a519-49c9-befa-cfb88f7d9b38 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:00.889959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1814aa8-a575-4be3-bcc8-816700370480 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:00.914936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3751-d32f-4856-9737-8724f1f56121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:00.916614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3751-d32f-4856-9737-8724f1f56121 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.917410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3751-d32f-4856-9737-8724f1f56121 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.917897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3751-d32f-4856-9737-8724f1f56121 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.919860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3751-d32f-4856-9737-8724f1f56121 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:00.921974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f3751-d32f-4856-9737-8724f1f56121 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:00.925696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 147/296] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:00 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:00.928021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388cf8c-7385-4ef8-9bb1-0ee329f88e9b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.928535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388cf8c-7385-4ef8-9bb1-0ee329f88e9b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.929094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388cf8c-7385-4ef8-9bb1-0ee329f88e9b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.931060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 148/297] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:00.932087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97eb84-6374-48a7-b2e7-c95d23a77168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:00.932294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388cf8c-7385-4ef8-9bb1-0ee329f88e9b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:00.932953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97eb84-6374-48a7-b2e7-c95d23a77168 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:00.933305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97eb84-6374-48a7-b2e7-c95d23a77168 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:00.933645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97eb84-6374-48a7-b2e7-c95d23a77168 None admin] PATH_INFO: `/v3/groups` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:00.934455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97eb84-6374-48a7-b2e7-c95d23a77168 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:00.936233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97eb84-6374-48a7-b2e7-c95d23a77168 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:00.998051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.038033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 150/298] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Jun 16 17:47:00 2022] POST /identity/v3/groups => generated 229 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:01.040272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.041011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.041505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.050177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388cf8c-7385-4ef8-9bb1-0ee329f88e9b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.073760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:01.076198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:01.099382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.100122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.100411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.100691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.101894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=4454b5da82ad43bca0694b902113418a) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:01.102211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:01.104225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed7bc9-b6d5-4b2a-b1da-6b5514e69d49 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:01.113817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 149/299] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:00 2022] GET /identity/v3/role_assignments?user.id=4454b5da82ad43bca0694b902113418a&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=4ef493d63de24283a7dff2172a9c856c => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:01.115192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84283976-9649-4a11-8cc6-4722a7d092f7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.115554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84283976-9649-4a11-8cc6-4722a7d092f7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.116377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84283976-9649-4a11-8cc6-4722a7d092f7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.118350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 150/300] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:01.119240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.125024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.125448 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.125828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.167037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:01.168809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:01.185112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 151/301] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:01.231350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.282751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 151/302] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:01.309778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.372300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.392167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d679e4b-b55f-4a13-9ed1-901300e3597e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.432986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.433727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.434012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.434327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.437078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df806c-da3f-4f7a-814e-e148d6fe9117 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.448760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:01.457458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:01.458448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:01.459241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 152/303] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:01.464366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.477313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.478175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.478461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.478741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.489773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.493293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:01.501562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:01.503752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:01.504654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:01.506880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 152/304] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:01.514350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.590401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b646d-a1a6-434d-8611-7d52913dd734 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.630609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.631520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.631907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.632285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.633435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:01.633435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:01.635052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:01.644345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 153/305] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:01.648438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386b0a6-2593-4645-84c8-9eea22d14b54 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.649935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.660142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.694837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.694837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.695107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.695411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.696657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:01.696657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:01.698027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:01.709428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 153/306] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:01.715336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.776213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5ed0-1c99-4d0c-a533-e16785d7100b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.807772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.813585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd45e0-4134-49fe-ab40-bd18fa7089e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.814540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd45e0-4134-49fe-ab40-bd18fa7089e6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.814832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd45e0-4134-49fe-ab40-bd18fa7089e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.816121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd45e0-4134-49fe-ab40-bd18fa7089e6 None admin] PATH_INFO: `/v3/users/ea0ee62d776d48f09740e669c58b6154` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.830144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd45e0-4134-49fe-ab40-bd18fa7089e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ea0ee62d776d48f09740e669c58b6154)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:01.831710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd45e0-4134-49fe-ab40-bd18fa7089e6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:01.833413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 154/307] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/users/ea0ee62d776d48f09740e669c58b6154 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:01.834780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa6f84-138b-4481-93f6-eedb614a4ba5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.835069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa6f84-138b-4481-93f6-eedb614a4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.835346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa6f84-138b-4481-93f6-eedb614a4ba5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.836605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 155/308] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:01.840214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa6f84-138b-4481-93f6-eedb614a4ba5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:01.848328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eef725-7a7c-466a-9453-c23345813674 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.893974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460f84-f81f-47fb-b5ab-31c526cb0743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.894943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460f84-f81f-47fb-b5ab-31c526cb0743 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.895257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460f84-f81f-47fb-b5ab-31c526cb0743 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.895569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460f84-f81f-47fb-b5ab-31c526cb0743 None admin] PATH_INFO: `/v3/users/e79e9ad9fa434b00a3d1340ff07a76d0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.896812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460f84-f81f-47fb-b5ab-31c526cb0743 None admin] RBAC: Authorizing `identity:get_user(user_id=e79e9ad9fa434b00a3d1340ff07a76d0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:01.898267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460f84-f81f-47fb-b5ab-31c526cb0743 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:01.899927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 154/309] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/users/e79e9ad9fa434b00a3d1340ff07a76d0 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:01.905638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.905927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.906205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.922312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:01.923547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:01.951465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa6f84-138b-4481-93f6-eedb614a4ba5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:01.953803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:01.989993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:01.990897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:01.991383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:01.991808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:01.993069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] RBAC: Adding query filter params (role.id=3ce88f0555894d2690078253590cdce7, user.id=ea0ee62d776d48f09740e669c58b6154, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:01.993621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:01.994529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 155/310] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:01.995408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:01.996363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:02.001950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 156/311] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jun 16 17:47:01 2022] GET /identity/v3/role_assignments?user.id=ea0ee62d776d48f09740e669c58b6154&scope.system=all&role.id=3ce88f0555894d2690078253590cdce7 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:02.020341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:02.031621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:02.147174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabd1ac-13ea-4333-98e3-2bb2b415c5eb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:02.164658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a9e6d-fe57-466f-a918-0cc2b1118186 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:02.197967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:02.199385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:02.199385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:02.199385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] PATH_INFO: `/v3/system/users/e79e9ad9fa434b00a3d1340ff07a76d0/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:02.201097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e79e9ad9fa434b00a3d1340ff07a76d0, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:02.202966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:02.215476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:02.216276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:02.216559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:02.216836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:02.227983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:02.235927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:02.236821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:02.238220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:02.242622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 157/312] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:02.243515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:02.316775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 156/313] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Jun 16 17:47:01 2022] PUT /identity/v3/system/users/e79e9ad9fa434b00a3d1340ff07a76d0/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:02.324055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:02.367690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393592b-9797-42fd-98e2-6b55bac1276e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:02.419719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:02.420410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:02.420701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:02.420978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:02.421749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:02.422060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:02.423485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:02.434096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 158/314] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:02.439994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:02.475405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:02.556717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e34a7a-bded-476f-a6c6-26d81307b95d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:02.592690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:02.593847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:02.594277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:02.594693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] PATH_INFO: `/v3/users/6a59a8a18b1a4ee985cfa7060b16b29b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:02.596202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] RBAC: Authorizing `identity:get_user(user_id=6a59a8a18b1a4ee985cfa7060b16b29b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:02.597813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:02.599949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 157/315] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:47:02 2022] GET /identity/v3/users/6a59a8a18b1a4ee985cfa7060b16b29b => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:02.607420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:02.711876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e59e93-a491-43aa-9356-c48919b6b352 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:02.740345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:02.748850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:02.754849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:02.754849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:02.754849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:02.820846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:02.822332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:02.931480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 159/316] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:02.933166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:03.112632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:03.115568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb038afc-7311-456a-b7eb-133de0b5350e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:03.153169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e94469-a8f8-4bbe-aa78-a9b408654624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:03.154049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94469-a8f8-4bbe-aa78-a9b408654624 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:03.154333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94469-a8f8-4bbe-aa78-a9b408654624 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:03.154609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e94469-a8f8-4bbe-aa78-a9b408654624 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:03.155891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e94469-a8f8-4bbe-aa78-a9b408654624 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:03.157336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e94469-a8f8-4bbe-aa78-a9b408654624 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:03.159498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 158/317] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:02 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:03.165766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:03.195764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:03.273336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93763773-1e34-4bb1-8e38-a398b9b3f586 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:03.308876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:03.309544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:03.309828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:03.310108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:03.310933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] RBAC: Adding query filter params (role.id=3ce88f0555894d2690078253590cdce7, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1, user.id=6a59a8a18b1a4ee985cfa7060b16b29b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:03.311290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:03.312828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e1c92-6f5e-469f-b81e-2dfacd9613aa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:03.319786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 160/318] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:03 2022] GET /identity/v3/role_assignments?user.id=6a59a8a18b1a4ee985cfa7060b16b29b&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=3ce88f0555894d2690078253590cdce7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:03.468074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec05ff-f00e-4c20-95ae-68a1d8fd3dce None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:03.468635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec05ff-f00e-4c20-95ae-68a1d8fd3dce None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:03.468635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec05ff-f00e-4c20-95ae-68a1d8fd3dce None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:03.470344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 159/319] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:03.476793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:03.477303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:03.477492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:03.496485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:03.497845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:03.578988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 161/320] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:03.602587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec05ff-f00e-4c20-95ae-68a1d8fd3dce None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:03.709404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec05ff-f00e-4c20-95ae-68a1d8fd3dce None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:03.743873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197a92b-7be9-415b-ac8c-930e15257758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:03.744595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197a92b-7be9-415b-ac8c-930e15257758 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:03.744883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197a92b-7be9-415b-ac8c-930e15257758 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:03.745165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197a92b-7be9-415b-ac8c-930e15257758 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:03.746326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197a92b-7be9-415b-ac8c-930e15257758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:03.747795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197a92b-7be9-415b-ac8c-930e15257758 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:03.749594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 160/321] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:03.755195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:03.779041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:03.857886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208d7e2-411d-4399-8dbc-1e5d76945766 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:03.891991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a7c4-40a6-4a48-9502-660f9c734cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:03.892838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a7c4-40a6-4a48-9502-660f9c734cd9 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:03.893927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a7c4-40a6-4a48-9502-660f9c734cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:03.894360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a7c4-40a6-4a48-9502-660f9c734cd9 None admin] PATH_INFO: `/v3/groups` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:03.895149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05a7c4-40a6-4a48-9502-660f9c734cd9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:03.896721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05a7c4-40a6-4a48-9502-660f9c734cd9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:04.008851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 162/322] 10.222.0.58 () {60 vars in 1134 bytes} [Thu Jun 16 17:47:03 2022] POST /identity/v3/groups => generated 226 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:04.240852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd67ff-992d-4510-88d6-ea8e0173e945 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.241231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd67ff-992d-4510-88d6-ea8e0173e945 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.241410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd67ff-992d-4510-88d6-ea8e0173e945 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.244763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 161/323] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:04.248136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.248426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.248701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.267766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:04.269760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:04.343454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 163/324] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:04.365895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd67ff-992d-4510-88d6-ea8e0173e945 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:04.405990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:04.477725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd67ff-992d-4510-88d6-ea8e0173e945 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:04.513166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404c1f-f8fd-47f9-9a75-1c527d74bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:04.514120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404c1f-f8fd-47f9-9a75-1c527d74bd35 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.514403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404c1f-f8fd-47f9-9a75-1c527d74bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.514698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404c1f-f8fd-47f9-9a75-1c527d74bd35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.526127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404c1f-f8fd-47f9-9a75-1c527d74bd35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6d404c1f-f8fd-47f9-9a75-1c527d74bd35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:04.534323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:04.535228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:04.536083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 162/325] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:04.542904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:04.558326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:04.661463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aea25a-b4f0-485b-bf55-ce449c769da4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.661920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aea25a-b4f0-485b-bf55-ce449c769da4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.662055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829325e-25af-4071-85f0-6ede5dcc046b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:04.662394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aea25a-b4f0-485b-bf55-ce449c769da4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.663887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 163/326] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:04.669077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.669466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.669897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.688737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:04.690256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:04.712655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:04.713342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.713626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.713901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.714658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:04.714968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:04.716406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:04.729828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 164/327] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:04.735207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:04.778956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 164/328] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:04.813330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:04.844783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:04.848536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:04.882458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab665f5-03d9-458a-84ce-d8e811de6f8f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:04.933477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:04.934424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:04.934702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:04.934973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] PATH_INFO: `/v3/users/ea0ee62d776d48f09740e669c58b6154` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:04.936350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] RBAC: Authorizing `identity:get_user(user_id=ea0ee62d776d48f09740e669c58b6154)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:04.937814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:04.939632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 165/329] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:47:04 2022] GET /identity/v3/users/ea0ee62d776d48f09740e669c58b6154 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:04.945466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:04.986870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93a48-09f9-40a8-a20a-a1f0c97d0ed2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:05.053527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef565e-8550-4670-9601-d838acddcece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:05.054251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef565e-8550-4670-9601-d838acddcece None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:05.054531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef565e-8550-4670-9601-d838acddcece None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:05.054802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef565e-8550-4670-9601-d838acddcece None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:05.070055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef565e-8550-4670-9601-d838acddcece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-91ef565e-8550-4670-9601-d838acddcece None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:05.077728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:05.078625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:05.079477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:05.080084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 165/330] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:05.085862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef565e-8550-4670-9601-d838acddcece None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:05.215922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:05.230312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef565e-8550-4670-9601-d838acddcece None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:05.286414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:05.287087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:05.287391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:05.287669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:05.288423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:05.288732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:05.290117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:05.294590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095b45a-e892-4994-b250-50cbfb134349 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:05.314823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 166/331] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:05.337553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:05.371650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:05.372918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:05.373405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:05.373883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] PATH_INFO: `/v3/system/users/ea0ee62d776d48f09740e669c58b6154/roles/3ce88f0555894d2690078253590cdce7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:05.375689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ea0ee62d776d48f09740e669c58b6154, role_id=3ce88f0555894d2690078253590cdce7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:05.377305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:05.471549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 166/332] 10.222.0.58 () {58 vars in 1334 bytes} [Thu Jun 16 17:47:04 2022] PUT /identity/v3/system/users/ea0ee62d776d48f09740e669c58b6154/roles/3ce88f0555894d2690078253590cdce7 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:05.473499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b795e9e-a855-4ff0-81e5-777f2f3a1639 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:05.519482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bc052-2bbe-4ff9-b6e0-cc59296ae31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:05.520370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bc052-2bbe-4ff9-b6e0-cc59296ae31b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:05.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bc052-2bbe-4ff9-b6e0-cc59296ae31b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:05.520922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bc052-2bbe-4ff9-b6e0-cc59296ae31b None admin] PATH_INFO: `/v3/users/e79e9ad9fa434b00a3d1340ff07a76d0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:05.522146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bc052-2bbe-4ff9-b6e0-cc59296ae31b None admin] RBAC: Authorizing `identity:get_user(user_id=e79e9ad9fa434b00a3d1340ff07a76d0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:05.523624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bc052-2bbe-4ff9-b6e0-cc59296ae31b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:05.525618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 167/333] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:47:05 2022] GET /identity/v3/users/e79e9ad9fa434b00a3d1340ff07a76d0 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:05.531105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:05.544812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:05.712449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601864b-d88a-49b2-bf33-11e0b6c68bad None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:05.762995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:05.763714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:05.764002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:05.764281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:05.765127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] RBAC: Adding query filter params (role.id=4ef493d63de24283a7dff2172a9c856c, user.id=e79e9ad9fa434b00a3d1340ff07a76d0, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:05.765442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:05.766880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9507a-a4b6-46bc-8742-3ce55c11ec44 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:05.783315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 167/334] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jun 16 17:47:05 2022] GET /identity/v3/role_assignments?user.id=e79e9ad9fa434b00a3d1340ff07a76d0&scope.system=all&role.id=4ef493d63de24283a7dff2172a9c856c => generated 510 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:07.258423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3e3c-21dd-4d50-954c-c55d0f7238fe None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.258927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3e3c-21dd-4d50-954c-c55d0f7238fe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.259025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3e3c-21dd-4d50-954c-c55d0f7238fe None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.260366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 168/335] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:07.279828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.280341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.280441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.308889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:07.310189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.425104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559230d-9593-4a01-9843-2a1c2f45870e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.425753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559230d-9593-4a01-9843-2a1c2f45870e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.426237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559230d-9593-4a01-9843-2a1c2f45870e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.429175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 169/336] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:07.434430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.434971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.435487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.437288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:07.458650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 168/337] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:07.467451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:07.468655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:07.482455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:07.499910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:07.623500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 170/338] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:07.689474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:07.695405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31544089-88ea-4e4f-a75b-edd6db784ddc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:07.742146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282fc9-ddc1-4929-82b3-629f6dacd10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:07.743265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282fc9-ddc1-4929-82b3-629f6dacd10a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.743729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282fc9-ddc1-4929-82b3-629f6dacd10a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.744136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282fc9-ddc1-4929-82b3-629f6dacd10a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.768776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282fc9-ddc1-4929-82b3-629f6dacd10a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-02282fc9-ddc1-4929-82b3-629f6dacd10a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:07.781787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:07.782916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:07.784545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:07.786729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 169/339] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:07.789762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac443d-7295-4ef9-b0cd-c33d92180b97 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.790225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac443d-7295-4ef9-b0cd-c33d92180b97 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.790652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac443d-7295-4ef9-b0cd-c33d92180b97 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.792451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 170/340] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:07.798627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac443d-7295-4ef9-b0cd-c33d92180b97 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:07.888570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a0ef0-aa19-44c0-9013-3f70b2959b53 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:07.891803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:07.940319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9824b4-97f0-4d4c-88ea-6e0037f1d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:07.942545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9824b4-97f0-4d4c-88ea-6e0037f1d165 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.944543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9824b4-97f0-4d4c-88ea-6e0037f1d165 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.945021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9824b4-97f0-4d4c-88ea-6e0037f1d165 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:07.968362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9824b4-97f0-4d4c-88ea-6e0037f1d165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8c9824b4-97f0-4d4c-88ea-6e0037f1d165 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:07.986573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:07.987717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:07.988761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:07.991173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 171/341] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:07.992942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac443d-7295-4ef9-b0cd-c33d92180b97 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:07.997012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:07.997477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:07.998017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.027519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:08.032336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:08.033786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:08.063887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:08.066535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.066977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.069016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.069977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:08.070440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:08.073854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:08.094921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 171/342] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:08.096221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:08.153404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 172/343] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:08.154651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:08.262338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beca91-fb14-4ec5-8df7-20a2ad18267a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:08.286946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:08.306448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:08.307403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.307833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.308322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.309290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:08.309782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:08.311351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:08.322542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 172/344] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:08.323688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:08.344800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6032a-da46-439b-ac94-f9ccd65a5ce6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:08.405791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:08.406465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.406754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.407036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.414761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:08.440345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99b12c-8f04-49da-86ce-dc8fb40d871e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:08.478091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:08.479337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.480414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.480850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.493287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:08.507003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:08.512201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:08.513081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:08.513695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:08.514485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:08.515957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 173/345] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:08.516957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:08.583904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:08.657868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe412c-7ec4-491f-80ea-9992ae0e6d37 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:08.696258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:08.697166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.697659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.698077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.711733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 173/346] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:08.712605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:08.761481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:08.803393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:08.805262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:08.849305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c13f8-675e-4a77-9268-360b36c870d2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:08.863737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:08.890740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:08.891522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.899953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.899953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.899953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:08.899953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:08.899953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1343f-8c76-4222-b314-a4b601fd510a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:08.905997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 174/347] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:08.907529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec20fd2-6adc-4c6d-9af8-c899c1d8dee5 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:08.907818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec20fd2-6adc-4c6d-9af8-c899c1d8dee5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:08.908097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec20fd2-6adc-4c6d-9af8-c899c1d8dee5 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:08.911522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 175/348] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:08.912498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec20fd2-6adc-4c6d-9af8-c899c1d8dee5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:08.947519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 174/349] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:08.948545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.062215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.067002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec20fd2-6adc-4c6d-9af8-c899c1d8dee5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.092753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878d66b-03b8-4267-b3fa-32fa1a1cd623 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.118478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08643975-e29e-4eb0-892d-fce85b221024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.118478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08643975-e29e-4eb0-892d-fce85b221024 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.118478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08643975-e29e-4eb0-892d-fce85b221024 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.118478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08643975-e29e-4eb0-892d-fce85b221024 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.118860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08643975-e29e-4eb0-892d-fce85b221024 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.118860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08643975-e29e-4eb0-892d-fce85b221024 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:09.118860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 176/350] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:09.118860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.118860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.118860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.133676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387c84a-45f6-4746-bcc5-9c119922ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.134719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387c84a-45f6-4746-bcc5-9c119922ca64 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.135149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387c84a-45f6-4746-bcc5-9c119922ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.135635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387c84a-45f6-4746-bcc5-9c119922ca64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.139035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:09.141156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:09.145261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.214188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387c84a-45f6-4746-bcc5-9c119922ca64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.216703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387c84a-45f6-4746-bcc5-9c119922ca64 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:09.230479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 177/351] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:09.231343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.341771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 175/352] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:09.343026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3544e2e-fe3c-4f23-8294-b2beca68e3b1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.343322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3544e2e-fe3c-4f23-8294-b2beca68e3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.343666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3544e2e-fe3c-4f23-8294-b2beca68e3b1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.344942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 176/353] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:09.346630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c658488-9bdd-4a09-b10f-fdb6f169048f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.347195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c658488-9bdd-4a09-b10f-fdb6f169048f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.347279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c658488-9bdd-4a09-b10f-fdb6f169048f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.349923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 177/354] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:09.349923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c658488-9bdd-4a09-b10f-fdb6f169048f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.359834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.367878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bcb7e-1749-46d4-8b9d-a71e90a4e9ca None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.410764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.411064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.411278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.411591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.412872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.413935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.416132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:09.419487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 178/355] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:09.419487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.484388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c658488-9bdd-4a09-b10f-fdb6f169048f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.523986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.524851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.525133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.525403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.533171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.535407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:09.537496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 178/356] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:09.538079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.547875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380623e4-d407-40ee-a25f-ac4e58e23009 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.593788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88430cf8-c983-4ef7-b1b4-5de8e15f02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.594868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88430cf8-c983-4ef7-b1b4-5de8e15f02d7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.595153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88430cf8-c983-4ef7-b1b4-5de8e15f02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.595593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88430cf8-c983-4ef7-b1b4-5de8e15f02d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.609755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88430cf8-c983-4ef7-b1b4-5de8e15f02d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-88430cf8-c983-4ef7-b1b4-5de8e15f02d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:09.618679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:09.619790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:09.620622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 179/357] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:09.621314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.621611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.621898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.643548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:09.644766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:09.657856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6423b28-5634-49d7-9309-a901325bb345 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.659757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.687299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.703412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77aeb4b-adf6-4cc5-815c-48fdac4ecd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.704277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77aeb4b-adf6-4cc5-815c-48fdac4ecd68 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.704568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77aeb4b-adf6-4cc5-815c-48fdac4ecd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.704851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77aeb4b-adf6-4cc5-815c-48fdac4ecd68 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.706809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77aeb4b-adf6-4cc5-815c-48fdac4ecd68 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.708477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77aeb4b-adf6-4cc5-815c-48fdac4ecd68 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:09.712420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 179/358] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:09.712420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.712420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.712420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.730125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:09.730590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 180/359] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:09.731345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:09.731539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.819746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 180/360] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:09.820475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.860776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbc19e-3123-452e-9c89-0a5c295806b8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.905917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.906812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.907096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.907454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.909098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.911199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:09.912166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 181/361] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:09.912855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:09.946152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.948764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107e896-0dac-4481-9d8c-5344c40c946a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:09.950634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:09.989505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:09.990184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:09.990462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:09.990750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:09.991628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=91b1159015f54fa786a5b4615137d807) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:09.991947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:09.993407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.000228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 181/362] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=4ef493d63de24283a7dff2172a9c856c => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.001473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.040856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc95e0-b114-444b-8eef-c40b9fc628ce None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.095127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.096715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.097041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.097638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.098518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:10.098833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.101572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.113963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 182/363] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.114832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.139164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24da19-6a8b-4521-8c86-56406abc199a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.191450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.192660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.193115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.193566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.194962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.196735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.199002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 182/364] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.199971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.234729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:10.249308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:10.261143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421b0b6-f858-43ce-83f8-ddd31de5d30b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.302367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.303602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.304090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.304580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.319082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:10.328218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:10.329121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:10.329923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 183/365] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:10.330728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.339202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b1d6-be16-4419-b719-40da3de9e927 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.385237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.385899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.386184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.386460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.397383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:10.405037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:10.406029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:10.406959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 183/366] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:10.407916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.450132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3761-19bc-47cd-b0e0-d48557810592 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.484300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.484960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.485243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.485520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.487041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:10.487409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.488859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.496024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 184/367] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=4ef493d63de24283a7dff2172a9c856c => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.497637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.514016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:10.517952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:10.528156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f4334-3a15-4bf1-b47f-8d1fb2aaddaa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.566843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.567580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.567869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.568146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.617163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09106a2d-61b1-40fb-b0df-f770b78847fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.647253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.648758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.655131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.655977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.656366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.656765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.657728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=89845870f208412e84a72ff363bae771, scope.project.id=91b1159015f54fa786a5b4615137d807) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:10.658234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.660595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.667353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 185/368] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=89845870f208412e84a72ff363bae771 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.668449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.757578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:10.765537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:10.784219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 184/369] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:10.784219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.791716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f37d-4cc9-4b20-ae05-bb69350cd761 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.861853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.862548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.862839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.863120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.863915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:10.864277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.865682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.882304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c287c-d3d6-4e41-baf3-022f61909dfc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:10.886767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 186/370] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.887779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:10.921526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:10.922220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:10.922504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:10.922783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:10.923647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:10.924017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:10.925699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:10.935128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 185/371] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:10.940212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:11.003851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:11.022777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42376b60-28e8-4992-b7ae-6f247168d766 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:11.055236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e6869-d069-4e44-8217-2c3e433425c3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:11.072122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:11.073003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:11.073290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:11.073571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:11.074846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:11.076322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:11.078036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 187/372] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:11.079874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:11.095774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:11.096731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:11.096929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:11.097084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:11.097282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] PATH_INFO: `/v3/users/ea0ee62d776d48f09740e669c58b6154` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:11.098628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] RBAC: Authorizing `identity:get_user(user_id=ea0ee62d776d48f09740e669c58b6154)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:11.100091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:11.103425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 186/373] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:47:10 2022] GET /identity/v3/users/ea0ee62d776d48f09740e669c58b6154 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:11.103425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:11.213242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f4474-5bcd-4704-8ca7-de515689a747 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:11.240332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:11.266826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:11.268603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:11.269111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:11.269589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:11.272014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:11.273749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:11.276039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 187/374] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:11 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:11.277093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:11.309355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a5bf6-2ef8-40e8-bd17-600a9fe00b4c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:11.399730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:11.400773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:11.401290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:11.401766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:11.403271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6645f7-c68e-4c25-abb9-9cd67917afec None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:11.479260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:11.480481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:11.481076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:11.481573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:11.482659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] RBAC: Adding query filter params (role.id=3ce88f0555894d2690078253590cdce7, user.id=ea0ee62d776d48f09740e669c58b6154, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:11.483174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:11.487299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:11.516629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 188/375] 10.222.0.58 () {56 vars in 1308 bytes} [Thu Jun 16 17:47:11 2022] GET /identity/v3/role_assignments?user.id=ea0ee62d776d48f09740e669c58b6154&scope.system=all&role.id=3ce88f0555894d2690078253590cdce7 => generated 510 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:11.529639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:11.616758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:11.619060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:11.623964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:11.706860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:11.898386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61391c5-ac94-4998-8b15-fe31ca70f371 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:11.965437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 188/376] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 887 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:11.966591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:12.028604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:12.030184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.030865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.031518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.032523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=89845870f208412e84a72ff363bae771, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:12.033652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:12.036777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:12.059178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 189/377] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:11 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=89845870f208412e84a72ff363bae771 => generated 260 bytes in 541 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:12.138911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd562d-38ee-4319-9776-d6f8a330ba19 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:12.140686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:12.183212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:12.184112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.184401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.184677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] PATH_INFO: `/v3/projects/24af7229251445019ca42ea240bb437e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.185871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] RBAC: Authorizing `identity:get_project(project_id=24af7229251445019ca42ea240bb437e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:12.187301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:12.190076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 189/378] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:11 2022] GET /identity/v3/projects/24af7229251445019ca42ea240bb437e => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:12.195445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:12.342437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4fe8b8-7ccb-4709-988f-a6590a80e151 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:12.361282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:12.387273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e3aaa-0825-436d-a5cc-5a8b3c8b704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:12.388567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3aaa-0825-436d-a5cc-5a8b3c8b704a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.389062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3aaa-0825-436d-a5cc-5a8b3c8b704a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.392111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3aaa-0825-436d-a5cc-5a8b3c8b704a None admin] PATH_INFO: `/v3/groups/3081fced84ca4476a119b2ed22c698fa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.400739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e3aaa-0825-436d-a5cc-5a8b3c8b704a None admin] RBAC: Authorizing `identity:get_group(group_id=3081fced84ca4476a119b2ed22c698fa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:12.402507 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e3aaa-0825-436d-a5cc-5a8b3c8b704a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:12.404831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 190/379] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:12 2022] GET /identity/v3/groups/3081fced84ca4476a119b2ed22c698fa => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:12.411763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:12.570278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139886c2-2de6-4243-b230-eb2862353769 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.570567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139886c2-2de6-4243-b230-eb2862353769 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.570761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139886c2-2de6-4243-b230-eb2862353769 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.571658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28119a61-4ed7-4827-8c84-c06c74a75d93 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:12.572315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 191/380] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:12.579010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.579302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.579616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.615231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:12.616468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:12.624191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:12.624856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.625139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.625415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.626249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] RBAC: Adding query filter params (group.id=3081fced84ca4476a119b2ed22c698fa, role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=24af7229251445019ca42ea240bb437e) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:12.626560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:12.629148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:12.641982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 190/381] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:12 2022] GET /identity/v3/role_assignments?group.id=3081fced84ca4476a119b2ed22c698fa&scope.project.id=24af7229251445019ca42ea240bb437e&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:12.741628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 192/382] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:12.776368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:12.822549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:12.896820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10bf9e-eadc-4433-8915-0a931bb2ce82 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:12.937584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:12.938390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:12.938677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:12.938958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:12.951853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:12.960266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:12.961161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:12.961945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 191/383] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:12.968801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:13.089711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc19e8e-3af0-4b95-8f42-e2418c93a8a8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:13.098224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:13.138487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:13.139176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.139487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.139770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.140538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:13.140846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:13.142229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:13.152511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 193/384] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:13.158685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:13.301313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:13.308604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565d96-c05a-4017-a5a0-49b86fa753b7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:13.359120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c876479-05ac-44dc-93d2-5211a62b003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:13.359940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c876479-05ac-44dc-93d2-5211a62b003a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.360225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c876479-05ac-44dc-93d2-5211a62b003a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.360504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c876479-05ac-44dc-93d2-5211a62b003a None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.361732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c876479-05ac-44dc-93d2-5211a62b003a None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:13.363171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c876479-05ac-44dc-93d2-5211a62b003a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:13.364835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 192/385] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:13 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:13.370470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:13.452414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165be0-12e8-45c3-bf9f-f9dd33884da2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.452866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165be0-12e8-45c3-bf9f-f9dd33884da2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.454218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165be0-12e8-45c3-bf9f-f9dd33884da2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.456959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 193/386] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:13.462880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.463812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.464303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.498527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:13.500201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:13.519973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7271e-0f5e-4436-afe4-3101e0c95e1d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:13.560317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b8719-c4e6-495b-ad88-83d3d69dc478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:13.561193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b8719-c4e6-495b-ad88-83d3d69dc478 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.561474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b8719-c4e6-495b-ad88-83d3d69dc478 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.561758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b8719-c4e6-495b-ad88-83d3d69dc478 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.562945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b8719-c4e6-495b-ad88-83d3d69dc478 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:13.565333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b8719-c4e6-495b-ad88-83d3d69dc478 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:13.567607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 194/387] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:13 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:13.569019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e2ce8-8f7e-4715-baf9-cc2395ef3de2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.569446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e2ce8-8f7e-4715-baf9-cc2395ef3de2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.569963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e2ce8-8f7e-4715-baf9-cc2395ef3de2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.571541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 195/388] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:13.577415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.577707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.577987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.595992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:13.597195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:13.627608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 194/389] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:13.627608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:13.637665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:13.687408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 196/390] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:13.687408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:13.784075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:13.796996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f172-1e4c-43b8-8a05-3a428987b812 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:13.806241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f881c-f201-4cf3-afb0-7c27182f6df8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:13.839080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:13.840256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.840713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.841334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.843523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] RBAC: Authorizing `identity:create_grant(project_id=91b1159015f54fa786a5b4615137d807, group_id=1471a3aa6fd649bf95b23c1983e7bb8e, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:13.845159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:13.850726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:13.851591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:13.851875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:13.852147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:13.866141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:13.873993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:13.874910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:13.876079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:13.879555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 197/391] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:13.880634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:13.910649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:13.929630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 195/392] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jun 16 17:47:13 2022] PUT /identity/v3/projects/91b1159015f54fa786a5b4615137d807/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:13.930273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.023303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c7932-11bd-409b-8a81-c7d593fef1e6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.073704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.074891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.075439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.075974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.091470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:14.102326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:14.103289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:14.104282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:14.106747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 198/393] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:14.114854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.193435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a599863-d345-43f9-bc54-37755b2f67fe None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.230280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.230954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.231273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.231573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.232350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:14.232656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.234027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.246066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:14.247003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 196/394] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:14.250095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.274471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f74f3a-7d7e-479d-926c-4095e348e444 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.274614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:14.320219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.320920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.321213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.321500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.322263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:14.322577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.324037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.346012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 199/395] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:14.356780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.367967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a1049-b059-4378-a8ab-244bbe0d57b5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.404460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.405876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.406169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.406449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.407693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.410013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.412082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 197/396] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:14 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:14.417599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.517288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:14.526951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49cb27-a4d8-4530-8b81-746c60dbf4f2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.578536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.579428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.579748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.580027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.583386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.583386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.585095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 198/397] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:14 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:14.590946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.610853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3c95c-42bc-42e0-b164-24a7e30ff345 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.675628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.677034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.677529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.678087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.681765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.683239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.685276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 200/398] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:14 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:14.696350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.705588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955055c-97f5-4627-b147-8dddc3631db4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.730185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:14.742888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.744063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.744531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.744950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.747071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=816e8e0d63834df6ab20ef551cf3d9a1, group_id=1471a3aa6fd649bf95b23c1983e7bb8e, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.748693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.820374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 199/399] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jun 16 17:47:14 2022] PUT /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:47:14.842858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1cfec-f9e3-40bf-a88d-08a04fd18309 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:14.885410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd2016-63d7-43a2-86f2-db7cd53cb0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:14.886537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2016-63d7-43a2-86f2-db7cd53cb0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:14.886968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2016-63d7-43a2-86f2-db7cd53cb0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:14.887438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd2016-63d7-43a2-86f2-db7cd53cb0b6 None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:14.888804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2016-63d7-43a2-86f2-db7cd53cb0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:14.890470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd2016-63d7-43a2-86f2-db7cd53cb0b6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:14.892528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 201/400] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:14 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:14.899098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:14.925085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:15.027412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64303dbb-8abb-4f56-831e-39be30d967d4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:15.052526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d435-0b48-4c4b-a950-f3eab7633c61 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.052836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d435-0b48-4c4b-a950-f3eab7633c61 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.053090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d435-0b48-4c4b-a950-f3eab7633c61 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.054882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 202/401] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:15.062283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.062599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.062845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.078197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe6e63-cf41-43aa-a065-0b4e8a1af50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:15.079511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe6e63-cf41-43aa-a065-0b4e8a1af50b None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.080035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe6e63-cf41-43aa-a065-0b4e8a1af50b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.080515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe6e63-cf41-43aa-a065-0b4e8a1af50b None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/89845870f208412e84a72ff363bae771` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.082234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:15.083606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:15.088273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe6e63-cf41-43aa-a065-0b4e8a1af50b None admin] RBAC: Authorizing `identity:create_grant(project_id=91b1159015f54fa786a5b4615137d807, group_id=1471a3aa6fd649bf95b23c1983e7bb8e, role_id=89845870f208412e84a72ff363bae771)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:15.090054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe6e63-cf41-43aa-a065-0b4e8a1af50b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:15.166863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 203/402] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:15.189526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:15.206479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:15.208551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 200/403] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jun 16 17:47:14 2022] PUT /identity/v3/projects/91b1159015f54fa786a5b4615137d807/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/89845870f208412e84a72ff363bae771 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:15.319290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001909fa-be6e-4a6e-8c61-98da910eb534 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.319644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001909fa-be6e-4a6e-8c61-98da910eb534 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.320025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001909fa-be6e-4a6e-8c61-98da910eb534 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.321661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 201/404] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:15.327061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.327352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.327658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.333995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c5f8-7063-44f6-ab04-6c125789b0fd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:15.364536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:15.366777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:15.383123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:15.384383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.384902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.385383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.404067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:15.415522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:15.416512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:15.417311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:15.420150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 204/405] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:15.427409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:15.478901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:15.484242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 202/406] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:15.509001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:15.524291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:15.550618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8b546-81fc-4386-84c1-7d5a54b536ed None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:15.614088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:15.614714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.614973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.615252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.616045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:15.616357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:15.618025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:15.629445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 205/407] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:15.634440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:15.660209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d41a5-de52-47ee-9578-b435900e3539 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:15.718606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:15.719471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.719813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.720095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.734897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:15.751388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:15.752345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:15.753145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 203/408] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:15.755201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:15.797697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9104a5-bf08-4b59-a908-0bb87ebd898f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:15.823648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:15.870359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eef83f-79f4-432c-b17a-50d6ebcd088d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:15.882717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:15.883614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.883903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.884180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.885471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:15.886924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:15.888585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 206/409] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:15.895470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:15.902007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:15.952670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:15.953357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:15.953643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:15.954002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:15.955279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:15.955629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:15.957202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:15.971525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 204/410] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:15.973138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:16.061427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9cdcd-0830-4150-966f-97be36b57ccb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:16.132010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:16.133360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.133872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.134312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.136074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:16.139218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:16.144387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 207/411] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:16.148371 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:16.156693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1265a-5212-4e56-90c0-fd448459c28f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:16.196250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:16.221847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:16.222722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.223013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.223293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] PATH_INFO: `/v3/groups/3081fced84ca4476a119b2ed22c698fa` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.224984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] RBAC: Authorizing `identity:get_group(group_id=3081fced84ca4476a119b2ed22c698fa)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:16.227091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:16.239713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 205/412] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:15 2022] GET /identity/v3/groups/3081fced84ca4476a119b2ed22c698fa => generated 229 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:16.242549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:16.287258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:16.358259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba554b6-d7a2-4242-b391-247b6ce1b97a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:16.425928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bdf34-ad89-40c9-8432-c3e03b2ea85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:16.426880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdf34-ad89-40c9-8432-c3e03b2ea85e None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.427164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdf34-ad89-40c9-8432-c3e03b2ea85e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.427464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bdf34-ad89-40c9-8432-c3e03b2ea85e None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/89845870f208412e84a72ff363bae771` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.432194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bdf34-ad89-40c9-8432-c3e03b2ea85e None admin] RBAC: Authorizing `identity:create_grant(project_id=816e8e0d63834df6ab20ef551cf3d9a1, group_id=1471a3aa6fd649bf95b23c1983e7bb8e, role_id=89845870f208412e84a72ff363bae771)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:16.433645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bdf34-ad89-40c9-8432-c3e03b2ea85e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:16.438410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4f5cf-9337-4272-ac83-11189b1c29fd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:16.477415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:16.478282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.478564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.478839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] PATH_INFO: `/v3/projects/24af7229251445019ca42ea240bb437e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.480109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] RBAC: Authorizing `identity:get_project(project_id=24af7229251445019ca42ea240bb437e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:16.481587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:16.487481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 206/413] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:16 2022] GET /identity/v3/projects/24af7229251445019ca42ea240bb437e => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:16.490329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:16.524410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 208/414] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jun 16 17:47:16 2022] PUT /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1/groups/1471a3aa6fd649bf95b23c1983e7bb8e/roles/89845870f208412e84a72ff363bae771 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:16.583561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:16.644550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e5c37-7415-419b-80b0-a2d1b7e1eadb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:16.676347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f0dc-8048-40e7-b435-a5dcec8d225e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.676842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f0dc-8048-40e7-b435-a5dcec8d225e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.676842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f0dc-8048-40e7-b435-a5dcec8d225e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.678361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 209/415] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:16.684421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.684710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.685002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.698875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:16.700120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:16.700408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:16.700681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] PATH_INFO: `/v3/projects/24af7229251445019ca42ea240bb437e/groups/3081fced84ca4476a119b2ed22c698fa/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:16.702766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] RBAC: Authorizing `identity:create_grant(project_id=24af7229251445019ca42ea240bb437e, group_id=3081fced84ca4476a119b2ed22c698fa, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:16.704244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:16.713239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:16.714761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:16.789498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 207/416] 10.222.0.58 () {58 vars in 1442 bytes} [Thu Jun 16 17:47:16 2022] PUT /identity/v3/projects/24af7229251445019ca42ea240bb437e/groups/3081fced84ca4476a119b2ed22c698fa/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:47:16.797245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 210/417] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:16.823074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:16.967562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e46ee-ba70-45b5-96e5-3ddf128b7e27 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:16.987733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:17.006345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:17.007153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:17.007467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:17.007755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:17.019777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:17.027575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:17.028470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:17.029349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 208/418] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:17.035203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:17.104045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:17.143647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd333107-c83e-48bd-86f4-619e5e5bf775 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:17.183282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:17.183987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:17.184275 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:17.184556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:17.185302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:17.185614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:17.186980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:17.201859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 211/419] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:17.208129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:17.362092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ec18-7715-425b-bebb-4ac2da53f04c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:17.440982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f502e7-b0ba-4901-bbb3-8dcbde9b5072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:17.442616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f502e7-b0ba-4901-bbb3-8dcbde9b5072 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:17.443338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f502e7-b0ba-4901-bbb3-8dcbde9b5072 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:17.443914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f502e7-b0ba-4901-bbb3-8dcbde9b5072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:17.514494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:17.545204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f502e7-b0ba-4901-bbb3-8dcbde9b5072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:17.546865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f502e7-b0ba-4901-bbb3-8dcbde9b5072 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:17.663179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 209/420] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:17.669021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:17.684179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35058fd8-d5ab-4496-a8b3-fa550177ac9b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:17.684710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35058fd8-d5ab-4496-a8b3-fa550177ac9b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:17.685134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35058fd8-d5ab-4496-a8b3-fa550177ac9b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:17.687154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 210/421] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:17.691760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:17.692213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:17.692635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:17.720502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:17.721894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:17.797022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 211/422] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:17.833552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:17.849196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddec84-ff56-434f-9f10-16c959070e74 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:17.863748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:17.874239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:17.885228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:17.886464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:17.886930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:17.887405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:17.888790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:17.890406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:17.892981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 212/423] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:17 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:17.898515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.005364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bca23-c766-471f-a653-b163c0dc442a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.009284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f82f9-3c3e-49bd-a37e-829be923e242 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.041457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.042719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.043188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.043771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.045197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:18.046865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:18.047286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.048212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.049378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.049868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.050837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 213/424] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:17 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:18.057364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.064936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:18.075968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:18.076967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:18.077783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 212/425] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:18.084084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.139966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:18.173660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:18.193280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e64de3-fcb7-408e-a39c-ab13b269234c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.207770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6998af4-e59f-4016-9d22-876eba5e8eff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.242061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.242840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.243199 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.243995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.245193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=91b1159015f54fa786a5b4615137d807) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:18.245582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:18.247127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fbb38e-f5cb-4f81-aae2-163a9a4e91d2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:18.252802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.253733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.254196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.254664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.255617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:18.256103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:18.257648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:18.267394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 214/426] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=4ef493d63de24283a7dff2172a9c856c => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:18.267394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eae455-0e60-4f84-a417-6be9a741f80c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.267394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eae455-0e60-4f84-a417-6be9a741f80c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.267394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eae455-0e60-4f84-a417-6be9a741f80c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.267394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 215/427] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:18.273091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.273512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.273901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.285721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 213/428] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:18.291139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.292962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:18.294317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:18.381401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 216/429] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:18.415865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.521097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:18.524312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0f2f-b0bf-4ae7-b6cc-7bf75132cf43 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.571549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:18.580969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.581627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.581913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.582193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.587942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e949e99-17ae-4332-b686-36839a09fe27 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.643394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.643916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.644203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.644478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.659929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:18.668779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:18.669658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:18.670477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:18.675724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 217/430] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:18.676014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:18.676719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.677997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:18.804826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862af624-380e-490f-8fec-bc14e5d02e75 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:18.821587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:18.828927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 214/431] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:47:18.838030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.848348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:18.850163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:18.850163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:18.850163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:18.850814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:18.852530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:18.855160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:18.866107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 218/432] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:18.871613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:18.881205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:18.969679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96399f4f-7d50-4080-9020-a01930235aff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.012795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.013906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.014324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.014700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.016059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.016402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174bfa9-70f8-481a-a0c6-53289f68b922 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.018252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.025130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 215/433] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:19.028302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.063185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32e4c9-fd47-46cf-ad8e-cbd2283cf267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.064109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32e4c9-fd47-46cf-ad8e-cbd2283cf267 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.064745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32e4c9-fd47-46cf-ad8e-cbd2283cf267 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.065238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32e4c9-fd47-46cf-ad8e-cbd2283cf267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.118514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.150354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32e4c9-fd47-46cf-ad8e-cbd2283cf267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.152412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32e4c9-fd47-46cf-ad8e-cbd2283cf267 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.164814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.169585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05931810-8ed4-4d7c-8b91-cf24dcc92c82 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.211998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.213292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.213292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.213858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.215181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.216865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.223457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 216/434] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:19.227726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.300453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 219/435] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:19.302221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d3743-ed6c-4e18-944d-f9e6ca9ddce7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.302564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d3743-ed6c-4e18-944d-f9e6ca9ddce7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.302851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d3743-ed6c-4e18-944d-f9e6ca9ddce7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.304303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 220/436] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:19.308488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d3743-ed6c-4e18-944d-f9e6ca9ddce7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.366017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0661af-d591-4305-9deb-8357e61d8647 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.392884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.401751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.410150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.410150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.410150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.410150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=4ef493d63de24283a7dff2172a9c856c, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:19.410150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.410150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a92596-b9b7-49f0-854c-a3ae48a4b6b1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.424011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 217/437] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=4ef493d63de24283a7dff2172a9c856c => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:19.424011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.424011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.424011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.431175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d3743-ed6c-4e18-944d-f9e6ca9ddce7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.433059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:19.434298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:19.465465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.486513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.487942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.488854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.489349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] PATH_INFO: `/v3/projects/91b1159015f54fa786a5b4615137d807` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.491635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] RBAC: Authorizing `identity:get_project(project_id=91b1159015f54fa786a5b4615137d807)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.494170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.497147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 221/438] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/projects/91b1159015f54fa786a5b4615137d807 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:19.502952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.526963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 218/439] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:19.552409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.632280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142d133-1b28-4a06-80ca-c5f161dcae7e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.670182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205fd9a-5952-411e-954f-6dce439fbbd7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.670375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b553e-b100-4042-88ea-d6b3a07a0e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.671205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b553e-b100-4042-88ea-d6b3a07a0e63 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.671522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b553e-b100-4042-88ea-d6b3a07a0e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.671813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b553e-b100-4042-88ea-d6b3a07a0e63 None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.673032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b553e-b100-4042-88ea-d6b3a07a0e63 None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.674757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b553e-b100-4042-88ea-d6b3a07a0e63 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.676433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 222/440] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:19.677770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a95f2-0f5c-4c56-8e5d-9596df74e2e9 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.678131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a95f2-0f5c-4c56-8e5d-9596df74e2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.678418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a95f2-0f5c-4c56-8e5d-9596df74e2e9 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.679995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 223/441] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:19.682179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a95f2-0f5c-4c56-8e5d-9596df74e2e9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.697596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.707387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a46bcf-4f42-4ec0-b812-6daa09ae3fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.708262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46bcf-4f42-4ec0-b812-6daa09ae3fbf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.710281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46bcf-4f42-4ec0-b812-6daa09ae3fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.710281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a46bcf-4f42-4ec0-b812-6daa09ae3fbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.718990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a46bcf-4f42-4ec0-b812-6daa09ae3fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b4a46bcf-4f42-4ec0-b812-6daa09ae3fbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:19.727679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:19.728633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 219/442] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:19.734323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.735057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.735057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.746932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:19.748112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:19.761938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.787121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a95f2-0f5c-4c56-8e5d-9596df74e2e9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.815575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 220/443] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:19.816717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.839655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.840760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.841049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.841332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.842237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=89845870f208412e84a72ff363bae771, scope.project.id=91b1159015f54fa786a5b4615137d807) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:19.842555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.844012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.850856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 224/444] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=91b1159015f54fa786a5b4615137d807&role.id=89845870f208412e84a72ff363bae771 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:19.851663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:19.929355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e13c-ae8f-4715-806f-0360e01bc55b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.938399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:19.965478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6c6b3-0086-4ec5-a9d2-d22e6a2d902f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:19.969033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:19.969721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:19.970666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:19.971107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:19.971904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:19.972218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:19.973629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:19.988645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 221/445] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:19.989826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.008150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.012888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.013784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.014073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.014349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.026446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:20.034458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:20.035632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:20.036488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:20.039402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 225/446] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:20.042569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.111011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee4a5f-0064-4497-974b-829004c2e294 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.154030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.155145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.155468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.155754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.170456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4c818-5bca-4dcc-b03b-90a4783dd5d2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.215541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.217237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.217521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.217796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.218548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:20.218880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.220890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.224440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.232143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 226/447] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:20.237654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.238545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.239358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.260806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.367161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 222/448] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:20.372205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cecd6-d0ca-4b03-89ba-1b0123d51c6a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.372599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.410180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.410836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.411123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.411439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.474432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e92b4-b772-4bcc-9545-bdc47e13b612 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.477315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.480848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.482314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.488453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.509691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.510548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.510896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.511113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] PATH_INFO: `/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.512380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] RBAC: Authorizing `identity:get_project(project_id=816e8e0d63834df6ab20ef551cf3d9a1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.513854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.516038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 223/449] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/projects/816e8e0d63834df6ab20ef551cf3d9a1 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:20.521442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.582937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 227/450] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:47:20.588872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.621277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a53ee-462e-47c0-bdb6-7ea7fe3cdd92 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.656817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.657668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.657943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.658219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] PATH_INFO: `/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.659425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] RBAC: Authorizing `identity:get_group(group_id=1471a3aa6fd649bf95b23c1983e7bb8e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.660861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.662485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 224/451] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/groups/1471a3aa6fd649bf95b23c1983e7bb8e => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:20.668306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.693221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.697001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa545dcf-aa9f-4a8f-94e7-ed71c7e13bab None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.718779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:20.730996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.731890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.732178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.732466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] PATH_INFO: `/v3/projects/24af7229251445019ca42ea240bb437e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.733648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] RBAC: Authorizing `identity:get_project(project_id=24af7229251445019ca42ea240bb437e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.735091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.737458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 228/452] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/projects/24af7229251445019ca42ea240bb437e => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:20.742860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:20.770240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc3579-a3f2-4fde-b3d3-910d03dfb76f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.804903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.805552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.805835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.806111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.806934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] RBAC: Adding query filter params (group.id=1471a3aa6fd649bf95b23c1983e7bb8e, role.id=89845870f208412e84a72ff363bae771, scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:20.807246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.808788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.815468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 225/453] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/role_assignments?group.id=1471a3aa6fd649bf95b23c1983e7bb8e&scope.project.id=816e8e0d63834df6ab20ef551cf3d9a1&role.id=89845870f208412e84a72ff363bae771 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:20.860230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600208a1-5b33-4dfe-ab8d-9b207ba6c6f9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:20.898540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5303b-9c1f-433b-93f1-42201b2a1288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:20.899435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5303b-9c1f-433b-93f1-42201b2a1288 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:20.899730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5303b-9c1f-433b-93f1-42201b2a1288 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:20.899982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5303b-9c1f-433b-93f1-42201b2a1288 None admin] PATH_INFO: `/v3/groups/3081fced84ca4476a119b2ed22c698fa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:20.901205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5303b-9c1f-433b-93f1-42201b2a1288 None admin] RBAC: Authorizing `identity:get_group(group_id=3081fced84ca4476a119b2ed22c698fa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:20.902663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5303b-9c1f-433b-93f1-42201b2a1288 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:20.904408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 229/454] 10.222.0.58 () {56 vars in 1180 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/groups/3081fced84ca4476a119b2ed22c698fa => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:20.911882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:21.017891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04879f-e27c-4998-a989-bbc99ffeea42 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:21.031247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:21.053389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:21.054047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:21.054337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:21.054619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:21.055507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] RBAC: Adding query filter params (group.id=3081fced84ca4476a119b2ed22c698fa, role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=24af7229251445019ca42ea240bb437e) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:21.055825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:21.057269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c803f-f229-453d-8737-6343ae88324e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:21.063203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 226/455] 10.222.0.58 () {56 vars in 1376 bytes} [Thu Jun 16 17:47:20 2022] GET /identity/v3/role_assignments?group.id=3081fced84ca4476a119b2ed22c698fa&scope.project.id=24af7229251445019ca42ea240bb437e&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:23.681789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5109-23b5-4894-86a3-4d55996af704 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:23.682640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5109-23b5-4894-86a3-4d55996af704 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:23.683070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5109-23b5-4894-86a3-4d55996af704 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:23.684861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 230/456] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:23.691082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cefa0d-f9a4-4de5-b801-e9cbf16d5e6a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:23.691960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cefa0d-f9a4-4de5-b801-e9cbf16d5e6a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:23.692433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cefa0d-f9a4-4de5-b801-e9cbf16d5e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:23.715413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-54cefa0d-f9a4-4de5-b801-e9cbf16d5e6a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:23.716927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-54cefa0d-f9a4-4de5-b801-e9cbf16d5e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:23.810153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 227/457] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:23.834465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5109-23b5-4894-86a3-4d55996af704 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:23.835473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15be8e5-7ae8-4a40-bb7c-6ad3266817b8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:23.835965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15be8e5-7ae8-4a40-bb7c-6ad3266817b8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:23.836454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15be8e5-7ae8-4a40-bb7c-6ad3266817b8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:23.838290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 228/458] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:23.843946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:23.844486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:23.845564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:23.873185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:23.874686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:23.887715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:23.988523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 229/459] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:24.010641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5109-23b5-4894-86a3-4d55996af704 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.036788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:24.053433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:24.054591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.055075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.055611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.069458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:24.071215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:24.083536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1130495c-c7ea-42d5-8a4e-fd579dca0074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:24.090157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:24.100798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 231/460] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dc45c-9f22-428c-93fd-279c0966bb9b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dc45c-9f22-428c-93fd-279c0966bb9b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.101675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dc45c-9f22-428c-93fd-279c0966bb9b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.103169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 232/461] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:24.104171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2dc45c-9f22-428c-93fd-279c0966bb9b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:24.173187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a051d8-f24e-4118-86fd-55fe7bf0aef2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.197765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:24.214780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2dc45c-9f22-428c-93fd-279c0966bb9b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.221418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:24.222226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.222514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.222795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.238004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:24.240392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:24.261101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:24.261784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.262072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.262355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.263093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:24.263419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:24.264809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5ad8034b-fc54-473d-833f-e52c6d8aa241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:24.265395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:24.266241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:24.267685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:24.267685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:24.267685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:24.267685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:24.269972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 230/462] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:24.271481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.272163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.272640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.279164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 233/463] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:24.279164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:24.303873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:24.306561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:24.384007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943f12f-a824-429b-a487-e6c57d148521 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.422964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:24.423812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.424360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.424646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.425392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:24.425705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:24.427105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:24.445643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 234/464] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:24.445913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:24.447708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 231/465] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:24.448757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:24.463877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:24.486430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:24.577233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1678d-7baf-4e8c-a66a-70f5167b9bf4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.610862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03710b18-ecf5-4cec-9b01-90b03a3884bb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.629527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:24.630712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.631173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.631892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.632794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:24.634827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:24.648400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d941be1-5eb7-4cd8-b395-8b28667f5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:24.649076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d941be1-5eb7-4cd8-b395-8b28667f5b1d None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.649362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d941be1-5eb7-4cd8-b395-8b28667f5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.649643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d941be1-5eb7-4cd8-b395-8b28667f5b1d None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.650344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d941be1-5eb7-4cd8-b395-8b28667f5b1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:24.651843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d941be1-5eb7-4cd8-b395-8b28667f5b1d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:24.780213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 235/466] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:47:24 2022] POST /identity/v3/users => generated 246 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:24.781049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:24.815388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 232/467] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:47:24 2022] POST /identity/v3/users => generated 248 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:24.815388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5191ce1-9809-48e1-958b-8ad474b3a089 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.815388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5191ce1-9809-48e1-958b-8ad474b3a089 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.815388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5191ce1-9809-48e1-958b-8ad474b3a089 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.815388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 233/468] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:24.818402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:24.818758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:24.819091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:24.855110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:24.857162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:24.921479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:24.991210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10345a0-4d9b-47d7-b8b9-dfe0ff66cdeb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:24.996397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 234/469] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:25.024397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:25.028365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:25.033702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:25.035843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.035843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.035843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.050687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:25.052179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:25.072879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:25.073779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:25.074632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:25.074632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:25.074632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:25.074632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:25.074632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 236/470] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:25.081945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:25.161233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f18d7-b7a1-44fe-ab48-29c7db8e2bea None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:25.200817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:25.201381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.201667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.201941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.214412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:25.216025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:25.233253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d440f3-051a-4406-ab1e-1311a6b377f4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:25.237965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:25.238820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:25.239720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:25.239720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:25.239720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:25.239720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:25.239720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 235/471] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:25.256266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:25.258660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:25.278816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:25.279553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.279846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.280531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.281277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:25.281592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:25.283083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:25.303227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 237/472] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:25.309615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:25.348684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:25.433361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc5b36-97d3-480f-a456-ec2a3a343d28 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:25.477249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb8a95-ff33-4824-9641-628de93218c2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:25.479342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:25.480857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.481148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.481430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.482164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:25.482475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:25.483913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48127df-e942-450d-a78a-989f3b9241a5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:25.498556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 236/473] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:25.500467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed3268-2579-4ee8-8ee1-4577d4d8a5c9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.501704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed3268-2579-4ee8-8ee1-4577d4d8a5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.501996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed3268-2579-4ee8-8ee1-4577d4d8a5c9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.504014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 237/474] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:25.504724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed3268-2579-4ee8-8ee1-4577d4d8a5c9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:25.521858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6534605-b8e0-4edd-a5b0-16754688d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:25.522620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6534605-b8e0-4edd-a5b0-16754688d395 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.522906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6534605-b8e0-4edd-a5b0-16754688d395 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.523345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6534605-b8e0-4edd-a5b0-16754688d395 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.524119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6534605-b8e0-4edd-a5b0-16754688d395 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:25.526321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6534605-b8e0-4edd-a5b0-16754688d395 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:25.617082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 238/475] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:47:25 2022] POST /identity/v3/users => generated 248 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:25.618879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.619181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.619494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.643704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:25.645124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed3268-2579-4ee8-8ee1-4577d4d8a5c9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:25.645255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:25.676249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:25.706061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:25.721124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:25.722743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:25.724258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:25.725817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:25.728213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:25.730840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:25.785903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 239/476] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:25.819638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:25.889886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 238/477] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:47:25 2022] POST /identity/v3/users => generated 249 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:47:25.951840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f439c-3439-4599-9c24-a658357e6dcf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:26.010946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:26.011592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:26.011871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:26.012141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:26.012901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:26.014300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:26.054062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.sql.core [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:47:26.054062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:47:26.054062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [parameters: {'id': '975b9c835dc5403b8a3ba674ac3333a5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:47:26.054062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() Jun 16 17:47:26.083061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:47:26.084037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application yield resource Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise self.value Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application session.commit() Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:47:26.084944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:47:26.085810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:47:26.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:47:26.087526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [parameters: {'id': '975b9c835dc5403b8a3ba674ac3333a5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:26.088322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:47:26.089969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:26.092306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 240/478] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jun 16 17:47:25 2022] POST /identity/v3/roles => generated 144 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:26.093688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:26.223821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb332-d79c-44b0-8918-7ddd310f175b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:26.275751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d0d4-b608-446b-bb0f-108a7e1cb33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:26.276901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d0d4-b608-446b-bb0f-108a7e1cb33e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:26.277374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d0d4-b608-446b-bb0f-108a7e1cb33e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:26.277826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d0d4-b608-446b-bb0f-108a7e1cb33e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:26.290046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5d0d4-b608-446b-bb0f-108a7e1cb33e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-79d5d0d4-b608-446b-bb0f-108a7e1cb33e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:26.299662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:26.300771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:47:26.301770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:26.304699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 239/479] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:47:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:26.311072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:26.355916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:26.466171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca36e0-a25d-4210-8de3-7c7411f31e55 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:26.509306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:26.510442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:26.510948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:26.511263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:26.514386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:26.514726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:26.516745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c58c4-1510-40c5-a31f-5f84ab97d8b8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:26.526368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 241/480] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:27.619146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d71f0c-69d3-4860-a7eb-efaa68e6e7fa None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:27.619655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d71f0c-69d3-4860-a7eb-efaa68e6e7fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:27.619655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d71f0c-69d3-4860-a7eb-efaa68e6e7fa None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:27.621272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 240/481] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:27.627039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b8f4a-2c2c-43f1-a8e1-3a60c955e889 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:27.627472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b8f4a-2c2c-43f1-a8e1-3a60c955e889 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:27.627639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b8f4a-2c2c-43f1-a8e1-3a60c955e889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:27.649072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bb2b8f4a-2c2c-43f1-a8e1-3a60c955e889 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:27.650251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bb2b8f4a-2c2c-43f1-a8e1-3a60c955e889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:27.753679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 242/482] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:27.769434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b18e1d-dee3-42aa-aef0-80cdcb62b454 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:27.769722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b18e1d-dee3-42aa-aef0-80cdcb62b454 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:27.770004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b18e1d-dee3-42aa-aef0-80cdcb62b454 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:27.771737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 241/483] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:27.777225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:27.777522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:27.777802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:27.790748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b18e1d-dee3-42aa-aef0-80cdcb62b454 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:27.809946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:27.811163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:27.831188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:27.839680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:27.919312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b18e1d-dee3-42aa-aef0-80cdcb62b454 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:27.928537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 243/484] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:27.961315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.005402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.006201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.006484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.006756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.025388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:28.034032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:28.034923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:28.035776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:28.039438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 242/485] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:28.041163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.151921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:28.161370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe8c04-a35c-4816-b1bf-a174afbaeb37 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.193521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf23662-3ae7-4d88-99a6-514bda68e9c6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.208438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.209121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.209403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.209676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.210424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:28.210736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:28.212130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:28.229247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 243/486] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:28.229744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.230685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.230986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.231273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.240367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.242824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:28.250924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:28.251939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:28.252764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 244/487] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:28.259623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.276490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:28.394081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05bcf1-69aa-4fbd-ad30-532a465272f7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.432313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.433108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.433397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.433675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.445243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:28.446606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:28.459956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80691e3-c0bb-4cb5-a2f5-8fd306d9f5ea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1a8c6d2e-757c-4ded-9253-a32cf4d4120b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:28.473043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:28.473931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 244/488] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda2365-7ad4-4608-8159-5c7048b32b89 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda2365-7ad4-4608-8159-5c7048b32b89 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.479493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda2365-7ad4-4608-8159-5c7048b32b89 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.481736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 245/489] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:28.482621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda2365-7ad4-4608-8159-5c7048b32b89 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.492605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:28.515723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.516640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.516750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.517010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.517761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:28.518068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:28.519452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61409ff-097d-47d2-badd-968029be5320 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:28.532221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 245/490] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:28.533912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.534205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.534482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.556801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:28.558517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:28.613977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda2365-7ad4-4608-8159-5c7048b32b89 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.636882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:28.650265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 246/491] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:28.650265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.667967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.668639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.670062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.670346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.671757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:28.672070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:28.674962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:28.690327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 246/492] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:28.691166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.783576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:28.788771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a76fea-9d92-466b-920a-9c5c44c9a2c8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.823156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3cf5f-687f-4ede-a4ed-8e90ac6d15be None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:28.836935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.837894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.838283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.838661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.850824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:28.852684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:28.865072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:28.866161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:28.866589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:28.867008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:28.873717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:28.874768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:28.875946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:28.875946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:28.875946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:28.875946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:28.877722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 247/493] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:28.878652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.881037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:28.891752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:28.892965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:28.893963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:28.897011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 247/494] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:28.897979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:28.946779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.008443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9289cf-52a3-495c-8d89-651ecfe9bf32 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.013353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597939d8-fb99-47a4-aaf1-c0ff0f662a11 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.041582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.045696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.046427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.046754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.047035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.051694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.052403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.052692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.052972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.053699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:29.055118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.055211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.055541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.056498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.068487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 248/495] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:29.072300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-90931d58-0a06-4bc1-92dc-0735566744ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:29.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:29.074820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:29.075970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 248/496] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:47:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:29.076690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4489-8b80-4593-8226-61952d8f0f3c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.076690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4489-8b80-4593-8226-61952d8f0f3c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.076832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4489-8b80-4593-8226-61952d8f0f3c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.078211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 249/497] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:29.079547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4489-8b80-4593-8226-61952d8f0f3c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.185540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0376e2-d393-4c2a-b8a3-d59963fdf9f5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.192587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4489-8b80-4593-8226-61952d8f0f3c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.205048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.222177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.222851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.223237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.223489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.224242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:29.224557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.225915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.229510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.230311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.230597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.230877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.234748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 249/498] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:29.234748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.240876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.244707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0fa7a81d-f14f-4607-bd36-80e6c2630a90 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:29.258192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:29.259104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:29.260058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 250/499] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:29.260625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.261303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.261303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.279126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:29.280359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:29.300797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.354748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c024df-c603-4a40-914f-d9241e80346b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.364587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 251/500] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:29.365264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.410070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.410746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.411037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.411311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.412188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:29.412503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.413926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.452295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 250/501] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:29.453133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.473475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.476557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f589b-19f4-4a59-8766-92725edaa3af None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.511891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.512684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.512971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.513248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.523166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.524657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:29.541925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:29.542827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:29.544219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:29.544219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:29.544219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:29.544219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:29.544219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 252/502] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:29.544219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.558211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d115ef-9f43-49f1-badb-667c232fc740 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.567923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.598124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.617546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.617546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.617546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.617546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:29.617546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.617546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204d5f9-a9b9-4179-9232-cca3f82793b9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.623544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 251/503] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:29.624884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f45ad-4745-4075-b693-1341de8ff760 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.625177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f45ad-4745-4075-b693-1341de8ff760 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.626128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f45ad-4745-4075-b693-1341de8ff760 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.627491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 252/504] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:29.628200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f45ad-4745-4075-b693-1341de8ff760 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.649146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fd537-a539-437b-a50e-543c2316e9d7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.684300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.685107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.685396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.685674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.696185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:29.704117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:29.705245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 253/505] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:29.711011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.721363 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.729352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f45ad-4745-4075-b693-1341de8ff760 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.763476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6f797-69e2-4640-8b60-fc89ebf80850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.764124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6f797-69e2-4640-8b60-fc89ebf80850 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.764409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6f797-69e2-4640-8b60-fc89ebf80850 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.764688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6f797-69e2-4640-8b60-fc89ebf80850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.795660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.805060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b2168-9e28-4d41-8f6c-2fe477f00556 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.833156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6f797-69e2-4640-8b60-fc89ebf80850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.834596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6f797-69e2-4640-8b60-fc89ebf80850 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.837488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.838153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.838437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.838714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.839511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:29.839841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:29.841170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:29.853332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 254/506] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:29.854124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:29.924368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:29.938394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 253/507] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:29.940053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.940349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.940629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:29.952912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4f9a9-f5f7-4ad9-8e83-c65e1828810b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:29.956771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:29.957976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:29.988148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:29.989332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:29.989332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:29.989332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.015512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.023945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 254/508] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:30.024758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.062018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.063684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.124529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91568fe1-c247-4f73-9919-05cdb161fa9f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.131510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.159034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.159740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.160028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.160312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.161054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:30.161364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.162748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.163048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 255/509] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:30.163918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.175842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 255/510] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:30.175842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.229996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.267869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f460-26cb-46e0-a6d9-63d52351f0fe None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.274501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13917-fa67-4710-80da-2c0cb9942449 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.301444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.302240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.302522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.302799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.308880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.309626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.309740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.309929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.310205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.311056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.320750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.322180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:30.325245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:30.326147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:30.327057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 256/511] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:30.327680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:30.339492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:30.340543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:30.341438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:30.341438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:30.341438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.341438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:30.341438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 256/512] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:30.341438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.363520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.431463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f26ad-33c6-4b7f-8b4d-809764a686bc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.443649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cf8c3-89df-4b9a-af3f-a91fabd6c70f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.452389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.465068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.470264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.470264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.470264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.478002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.478164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.478869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.479153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.479472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.480912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.490451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.492002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:30.506020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:30.506944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:30.507842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:30.507842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:30.507842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.507842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:30.507842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 257/513] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:30.508117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:30.510742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:30.511951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:30.513593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:30.513593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:30.513593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.513593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:30.514235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 257/514] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:30.515112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.593382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.617491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f1b1-726c-431d-a556-64c907574beb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.618278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418bad6-6b06-4f69-976a-ef0ddcebcb79 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.657832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.658509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.658796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.659074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.660028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:30.660263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.661695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.665566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.666369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.666655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.666952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.677838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.679307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.679660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 258/515] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:30.680465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.699648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:30.700781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:30.701793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:30.702656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:30.702656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:30.702656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:30.702656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:30.702656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 258/516] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:30.702928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.786421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393ac3d-1155-4bb3-bc69-d5c2607dcfbe None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.808673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0164-061c-43ac-b4a4-8eabca44e366 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.820861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.821528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.821816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.822097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.822823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:30.823191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.824618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.836523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 259/517] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:30.837510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.844226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.844905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.845192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.845477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.846205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:30.846543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.848010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.850583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.860661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 259/518] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:30.861474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.919563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:30.942398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd58e-da7e-494c-90cf-c3480b310a69 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.963073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6eb3fa-3734-4187-98c7-5917ceab5d7b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:30.977037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.981698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.981698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.981698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:30.981698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:30.981698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:30.981698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:30.998191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 260/519] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:30.998191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:30.998442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:30.998442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:30.998442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:30.998442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.062044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.069835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.079140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.091945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c24ae-93f0-49d0-8582-943983889a29 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.126516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.127256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.127604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.127889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.128696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:31.129009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.130382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.136753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.141958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 261/520] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:31.142750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.170998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 260/521] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:31.171883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.246333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556318bc-fb61-487c-abf7-7a1dbc44a0c5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.273295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.279905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8b38-6aec-40ff-99f1-a597d87284b6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.280343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.281188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.282015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.282879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.289601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.291028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:31.306564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:31.307634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 262/522] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:31.313674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.316034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.316710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.316994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.317267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.317964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.319395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.370405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.410579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97289573-55cd-4a0f-b355-0ae549074c85 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.444130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.444915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.445205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.445488 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.446813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 261/523] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:47:31 2022] POST /identity/v3/users => generated 247 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:31.447657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.453224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.455692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:31.471442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:31.473804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 263/524] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:31.474676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.569005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d1bc8-1750-4e4a-9e35-85eb2d326dbc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.599614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71477-9552-4378-ae35-1d00e0d270c4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.605322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.606137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.606485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.606767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.616587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.618210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.620724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:31.632705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:31.635803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 262/525] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:31.636765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.641754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.642567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.642851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.643153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.654372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.657238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.659401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:31.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:31.680606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:31.681669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:31.681669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:31.681669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:31.681669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:31.681669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 264/526] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:31.682008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.749298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537f98f-d268-4eda-8554-b434a55b3e31 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.784751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca653e-9eea-4ecf-b006-d0eb41b9f126 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.785926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.786718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.787129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.787425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.788227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:31.788539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.789980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.813184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 263/527] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:31.813184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.836596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.837991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.838288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.838566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.839358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:31.839696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.841207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:31.866134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 265/528] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:31.866134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:31.897416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.902809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:31.932878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e247f-e40a-44f9-924e-420b2bf27cf7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.966963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d22881-0423-4582-b193-107fd8f773f0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:31.967335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:31.968040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:31.968334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:31.968613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:31.969510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:31.969833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:31.971818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.006402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.007071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.007357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.007669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:32.008391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:32.008699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:32.014792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 264/529] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:32.014926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.015015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.026114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 266/530] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:32.026946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.125856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:32.129455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8236d-be98-419e-9362-c5712c6d16d0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:32.142236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de802b6-feec-4d9f-b0cf-8c60628f1c4c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:32.164488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:32.171649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.173149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.178020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.178020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:32.178020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=9279195b1ed64901a63a3fce8d0bb610) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:32.178020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:32.178301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.178402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.178473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.178473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.178473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:32.178856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=6f37017740874fef845f23fca4f977f4) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:32.179163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:32.180625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.191762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 265/531] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/role_assignments?user.id=9279195b1ed64901a63a3fce8d0bb610&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:32.191762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.192030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 267/532] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/role_assignments?user.id=6f37017740874fef845f23fca4f977f4&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:32.192030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.399385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed858e78-3c23-4261-a64d-7e8cee8daf8d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:32.402595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3c7b5-7418-481a-9a60-ab21da91ddd2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:32.457487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.458340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.458620 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.459319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:32.464644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.465346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.465631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.465904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:32.473163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:32.474606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:32.501695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:32.502607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:32.503489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:32.510669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 268/533] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:32.516931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.544362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:32.587391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:32.593096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:32.594767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.710225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b283fd-3216-42d6-9ced-e10fe22336ac None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:32.761615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 266/534] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:32.767101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.776287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.777285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.777764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.778309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:32.779298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:32.779826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:32.782181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:32.801334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 269/535] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:32.806633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:32.919311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:32.930569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd9752-fae1-46b7-95a6-0ad4ba97c2b2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:32.974529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:32.979178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:32.980411 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:32.981666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:32.982157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.001508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:33.003975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:33.019123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcf5df-d83a-48d9-8e0d-ed23b8d23060 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:33.039475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:33.040406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:33.041236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:33.041236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:33.041236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:33.041236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:33.043519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 267/536] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:33.052730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:33.104622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:33.105663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.106148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.106643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.107717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=fbc41720695b45ec84c2f250238ac37b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:33.108228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:33.111175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:33.126469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 270/537] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:32 2022] GET /identity/v3/role_assignments?user.id=fbc41720695b45ec84c2f250238ac37b&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:33.164432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dacd2-bed0-4d8b-a7e7-0399d4f60e8f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:33.237601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:33.238206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.238495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.238775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.239536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:33.239847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:33.241239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:33.278693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 268/538] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:33.285445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:33.425958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb173cc7-dd37-4a40-9e70-17e2df7285bb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:33.489152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:33.490358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.490848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.491355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.508880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:33.511635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:33.516275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:33.536565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:33.542424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:33.543276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:33.545712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 271/539] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:33.551943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:33.563737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:33.703107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563a69e-c89a-4a2a-91be-79e582139a98 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:33.749257 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:33.749853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.750662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.750945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.751777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:33.752089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:33.753536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2899568-3df7-4452-bc21-af0ce545ddb3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:33.769331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 269/540] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:33.775049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:33.907272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13261e-70e5-4852-8bfb-13aa8d348104 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:33.950855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:33.951540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.951826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.952105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.952938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=6aae965af8284363820254c9286adf96) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:33.953477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:33.954757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a1d54-4e96-4211-8898-7db15ddec6ad None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:33.960868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 272/541] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:33 2022] GET /identity/v3/role_assignments?user.id=6aae965af8284363820254c9286adf96&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:33.983245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806befd-f689-4279-9ca8-e8f2238fe366 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.990348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806befd-f689-4279-9ca8-e8f2238fe366 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.990348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806befd-f689-4279-9ca8-e8f2238fe366 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:33.990348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 270/542] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:33.992274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:33.993109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:33.993602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:34.013015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:34.014526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:34.136095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:34.153228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 273/543] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:34.181162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806befd-f689-4279-9ca8-e8f2238fe366 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:34.284783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:34.345844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806befd-f689-4279-9ca8-e8f2238fe366 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:34.394890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:34.395728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:34.396011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:34.396289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:34.409783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:34.417612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:34.418502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:34.419297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 271/544] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:34.425418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:34.549264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe34374-6f91-4761-a381-6cc4b38fc8c6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:34.595200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:34.596262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:34.596791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:34.597288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:34.598399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:34.598913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:34.600512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:34.611044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 274/545] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:34.616411 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:34.742033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947b2d-4bc7-49c1-86fc-20e7ef22a951 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:34.769654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:34.784781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:34.785594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:34.785871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:34.786146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:34.798356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:34.800180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-97f14c15-a210-4536-96d6-e631a5f74e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:34.834464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:34.835512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:34.836378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:34.836378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:34.836378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:34.836378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:34.836378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 272/546] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:34.841442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:34.916906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b615a3-6555-4b95-ab0e-e7d99db21c02 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:34.917203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b615a3-6555-4b95-ab0e-e7d99db21c02 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:34.917448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b615a3-6555-4b95-ab0e-e7d99db21c02 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:34.920399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 273/547] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:34.925852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528103b-95be-47ce-9d99-2557bd81c032 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:34.926140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528103b-95be-47ce-9d99-2557bd81c032 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:34.926415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528103b-95be-47ce-9d99-2557bd81c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:34.934236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:34.944657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5528103b-95be-47ce-9d99-2557bd81c032 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:34.945850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5528103b-95be-47ce-9d99-2557bd81c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:35.003202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162089-d510-4420-9ee9-2a544e76723a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.029603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 274/548] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:35.032178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40482d52-9243-48b1-a8f2-ba8a69e44ac2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.032802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40482d52-9243-48b1-a8f2-ba8a69e44ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.033271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40482d52-9243-48b1-a8f2-ba8a69e44ac2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.035178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 275/549] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:35.040684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.041002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.041441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.044504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.045176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.045468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.045755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.046486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:35.047457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.049318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.064018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 275/550] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:35.072399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.073126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:35.074341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:35.149464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 276/551] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:35.150468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.184646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d7e44-8989-480b-994b-a9fe3073f7f2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.195196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:35.213194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:35.235716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.236582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.236873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.237254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.246549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.247621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:35.271765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:35.272747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:35.273848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:35.274383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6360d-a037-471e-bc1f-dd81fc09754d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.274456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 276/552] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:35.274456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.315516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.316249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.316536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.316812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.331104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:35.341195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:35.342320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:35.344151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:35.351386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 277/553] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:35.351386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.392363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f97d0c-80b6-476b-9a1e-5876eb79241d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.434266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.435326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.435740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.436124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.447839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:35.456684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:35.458563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:35.460435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 277/554] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:35.467391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.470355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:35.474075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae807a-9978-44d2-8a93-c606c3376199 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.507522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:35.510300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.511064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.511449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.511811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.512743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:35.513124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.514699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.538735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 278/555] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:35.538735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.606446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f94d-27c5-4c04-8736-8ce7a78fbe8c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.644406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.645227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.645605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.645952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.646767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:35.647145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.648647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.657711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c6ac5-41fd-408c-ab9a-533a5cc37785 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.664241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 278/556] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:35.665956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.700944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.702138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.702510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.703149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.703985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:35.704296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.706049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.724355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 279/557] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:35.725674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.755956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:35.785439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:35.791725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999667a-7ba3-4f88-a44e-9882cd6045fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.828743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b847c-e1d9-4532-b4ed-915eedaeacaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.829436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b847c-e1d9-4532-b4ed-915eedaeacaa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.829727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b847c-e1d9-4532-b4ed-915eedaeacaa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.830005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b847c-e1d9-4532-b4ed-915eedaeacaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.846013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd22200-dc88-4383-8061-386f5cc561bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:35.880829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:35.881547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:35.881834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:35.882112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:35.893213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.894722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.911088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b847c-e1d9-4532-b4ed-915eedaeacaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:35.912571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b847c-e1d9-4532-b4ed-915eedaeacaa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:35.916306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:35.917197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:35.918104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:35.918104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:35.918104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:35.918104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:35.925590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 280/558] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:35.925590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:35.995226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.018806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.027807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9750d1-66f5-4d7e-98e0-fdf1176b23a4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.031813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 279/559] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:36.033124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d350ac-0d07-4c59-aa8b-29cbbf6b1266 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.033417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d350ac-0d07-4c59-aa8b-29cbbf6b1266 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.033707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d350ac-0d07-4c59-aa8b-29cbbf6b1266 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.035032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 280/560] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:36.035763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d350ac-0d07-4c59-aa8b-29cbbf6b1266 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.084647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.085448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.085736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.086027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.097974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.099456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:36.117391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:36.118328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:36.119220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:36.119220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:36.119220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:36.119220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:36.119220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 281/561] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:36.119870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.164280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d350ac-0d07-4c59-aa8b-29cbbf6b1266 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.200490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.201153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.201436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.201707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.202423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:36.202732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.204968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b524c9-7f2d-417b-85ce-94abc9ade4e6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.217324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 281/562] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:36.218370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.219342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.220071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.235935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067f672-7abe-4582-890b-0475bf6c9ed0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.236698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:36.237894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:36.267292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.278263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.279143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.279588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.279937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.291015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.292579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.295629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:36.311235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:36.318688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:36.320067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:36.320067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:36.320067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:36.320067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:36.320067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 282/563] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:36.320067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.320395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 282/564] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:36.320395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.429612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4e5d6-6433-4f5d-88ba-5684753a2971 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.432161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3775c-a54d-4422-8b48-43c6e3138b67 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.470789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.471646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.471928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.472202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.472454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.473135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.473423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.473701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.474698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:36.475013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.477112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.480274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.481732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.495214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 283/565] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:36.495214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:36.497486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:36.498450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 283/566] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:36.499434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.546911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.553337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.607892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485196a0-8459-47be-8957-ee596edafe1f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.611960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474aada-e63d-4ca1-aa98-2c7b482773da None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.646781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.647474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.647762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.648040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.648757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:36.649069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.650105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.650429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.650902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.651185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.651491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.661999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:36.662155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 284/567] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:36.662873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:36.670160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:36.671100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 284/568] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:36.672225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.763650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.771273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a256d36-c70c-46c5-bc2e-c25fceb05c39 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.771837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fd2d1-1792-4764-bd07-dce0c0de1ca5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.778745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:36.804839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.805621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.805737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.805908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.806182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.806380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.806668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.806950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.807837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:36.808152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.809679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.814641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.816193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f4e36d3f-7407-4e49-a199-48b81945a5fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:36.832496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:36.833468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:36.834416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 285/569] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:36.835017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab42c6e-e0e3-4aa8-9e0a-4e9553dce698 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.835094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab42c6e-e0e3-4aa8-9e0a-4e9553dce698 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.835704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab42c6e-e0e3-4aa8-9e0a-4e9553dce698 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.838922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 286/570] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:36.838922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab42c6e-e0e3-4aa8-9e0a-4e9553dce698 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.842649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 285/571] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:36.843531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:36.940373 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab42c6e-e0e3-4aa8-9e0a-4e9553dce698 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.943008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583209b-7047-4680-8b03-b978b83ebe0f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:36.977456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.978285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.979548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.979676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:36.979771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.980222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:36.980516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:36.986910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:36.986910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:36.986910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.986910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.989484 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:36.991035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:36.992561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 286/572] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:36.993178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d5d89a2a-f64b-44dc-a5ce-658d99f01f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:37.008605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:37.009751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:37.010865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 287/573] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:37.011404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.011479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.011713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.030123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:37.031673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:37.036002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.041927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.107671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 288/574] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:37.107884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46db40-ee44-4b34-b10f-bfce071651c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.108467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.146827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.147524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.147808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.148086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.148912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.149221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.150627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.168363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 287/575] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:47:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.169180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.211860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72bfd0-5e82-403a-ab57-10ecdcc5d9dd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.245529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.246326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.246608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.246878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.254497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.259154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.260677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.263207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.269978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16af3a-cc05-4d5d-832c-0e037889cdab None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:37.278934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:37.280108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:37.286226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:37.286226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:37.286226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:37.286226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:37.286226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 289/576] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:37.286226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.306229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.307020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.307312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.307624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.318168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.319643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:37.337322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:37.338316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:37.339201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:37.339201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:37.339201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:37.339201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:37.339201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 288/577] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:37.339531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.396837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282ca5d-7b27-41f8-8722-4c6cde78e0a5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.430969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.431639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.431924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.432202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.432973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.433277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.434681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.442721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845d530-0b7a-40d5-9230-ae8cdaeb441f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.446625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 290/578] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.447445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.478104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.478914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.479198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.479502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.482106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.491105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:37.498698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:37.499812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:37.500670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 289/579] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:37.501375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.507652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.551854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dffdd8-6456-4a22-a594-57ac03524288 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.584567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.585354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.585640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.585924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.596763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.598226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.600770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df86f2a-cc40-474c-aa16-1cc15615006d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:37.615318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:37.616355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:37.617244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:37.617244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:37.617244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:37.617244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:37.617244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 291/580] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:37.617244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.638568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.639232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.639551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.639839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.641223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.641539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.642908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.654378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 290/581] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.655173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.712964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.717715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92453c-9f11-47a7-9d51-89af3bc7cd60 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.726597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.752507 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.753176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.753462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.753743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.754491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.754799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.756215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.757196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f267a-f405-4f88-8327-21350b09477f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.767892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 292/582] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.768694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.794580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.795231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.795554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.795834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.797161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=8bd57a7e4c074ca2890ec6bc081809cf) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.797551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.799146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.804787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 291/583] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/role_assignments?user.id=8bd57a7e4c074ca2890ec6bc081809cf&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.805386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.881031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408aff7-4a95-436b-a046-c512be102d4f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.914569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1a2d3-5c0a-46c5-ab94-ff90158e9247 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:37.920190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.920884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.921169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.921453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.922938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.923281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.924992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.940564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 293/584] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.940564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.945385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:37.961561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:37.962235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:37.962524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:37.962801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:37.963555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:37.963867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:37.965228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:37.978180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 292/585] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:37.979237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:37.979437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.053211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7532d-5661-4a03-a747-79fefed3f081 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.090446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.091183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.091522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.091783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.094867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367c39-de1d-4235-8075-112948310f78 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.099440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.100881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:38.115915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:38.116884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:38.117796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 294/586] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:38.118303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.136941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.137830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.138188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.138538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.146735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.148205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:38.162702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:38.163733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:47:38.164677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:38.167532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 293/587] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:38.169060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.213944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.219835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.230076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf49235-36f2-4163-9600-9891acd82936 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.272090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.273056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.273793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.274709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.280053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920bcf5-63a8-4886-896b-e8df5481f39e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.285708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.287158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:38.305680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:38.306754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:38.307878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:38.307878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:38.307878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:38.307878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:38.311472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 295/588] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:38.313265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.319631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.320453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.320743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.321024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.328702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.330208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:38.345190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:38.346063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:38.346954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 294/589] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:38.348333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.431826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f895c-357c-4320-a369-3db1ef67cd07 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.468634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.469301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.469587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.469867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.470674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:38.470787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.470977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.472880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.473414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5107a-f2e6-42e4-bc12-d494556421a5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.485387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 296/590] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:38.486203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.488770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.515381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.522038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.522038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.522038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.522038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:38.522038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.522038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.542016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 295/591] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:38.542016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.604576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea7e6-03cf-4827-88e5-2fc24c6ce3d8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.638617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.639283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.639600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.639884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.640704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:38.641013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.642377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.653181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94aeb-55e3-4ed6-92aa-2c785e7d8c32 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.654056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 297/592] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:38.654714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.700317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.700976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.701258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.701534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.702315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:38.702622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.704863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.716544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 296/593] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:38.717689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.726478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.743253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:38.784326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53608b-4cb0-4917-9867-55327149f01e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.820457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.821354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.821640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.821928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/6f37017740874fef845f23fca4f977f4/roles/777a339245484c7fbe89538db336a995` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.839502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=6f37017740874fef845f23fca4f977f4, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.840970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.853460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5d780-e2d4-49e0-bf4b-a1f76d39e014 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:38.887808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:38.888566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:38.888846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:38.889119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:38.901032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:38.902549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:38.910606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 298/594] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:47:38 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/6f37017740874fef845f23fca4f977f4/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:38.911763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:38.933633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:38.934619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:38.935557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:38.935557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:38.935557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:38.935557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:38.942122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 297/595] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:38.942122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.035973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:39.085341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36687ced-66aa-4a87-a304-b9faeffcf11e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.093008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d364a6-ee74-483a-9c23-337a31d0daaf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.109849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:39.131524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.132489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.132781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.133062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/9279195b1ed64901a63a3fce8d0bb610/roles/777a339245484c7fbe89538db336a995` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.146732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.148517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.149099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.149631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.150442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=9279195b1ed64901a63a3fce8d0bb610, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.152050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.159320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.160798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:39.177175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:39.178180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:47:39.179137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:39.181985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 299/596] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:39.183051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.233981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 298/597] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:47:38 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/9279195b1ed64901a63a3fce8d0bb610/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:39.240963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.351015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67519f2-fd66-4717-80fd-512c15247b6b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.436858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.438191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.438424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.438807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.439744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:39.440138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.441598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.454591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342d258-9245-4436-9cd3-ab9f71b41be0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.462104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 300/598] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:39.467126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.481575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:39.500653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.501582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.502022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.502506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.503674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:39.504134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.505806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.511535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:39.535199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 299/599] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Jun 16 17:47:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:39.536714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.588636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce903d-d0a5-45b8-a9ee-2756ee842f37 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.632469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.633439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.633829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.634210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.645938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.647437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:39.669259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:39.670229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:39.671118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 301/600] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:39.682307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.691952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944538ef-36b0-435f-be12-6970be3ac3f4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.755510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.756319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.756609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.757000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.780587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.782918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.795255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:39.805425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78b049-15c9-4036-b32b-e2a8f1b17a2a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:39.816722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:39.817673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:39.818554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:39.818554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:39.818554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:39.818554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:39.822642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 300/601] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:39.824019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.846391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.848014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.848285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.848565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.849361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:39.849673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.852506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:39.864273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:39.881404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 302/602] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:39.886624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:39.944844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b55299-156c-43bf-8ffb-ec28710b0e06 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:39.983309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:39.989273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:39.989273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:39.989273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:39.989273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:39.989273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:39.989273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.001141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 301/603] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:40.006811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:40.083716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a11ff4-0960-4ed9-9ade-296c0f474cad None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.122385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.123286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.123603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.123882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/fbc41720695b45ec84c2f250238ac37b/roles/777a339245484c7fbe89538db336a995` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.140594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=fbc41720695b45ec84c2f250238ac37b, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:40.142132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.174061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:40.184432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fb5a3-84ce-4626-a1f9-a65ddf5aac6f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.223776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.224517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.224852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.225130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.233067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:40.236404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1b388c5d-9dff-46d7-a522-93415f2290d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:40.257807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:40.258796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:40.260101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 302/604] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:40.260934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44de87-028e-44f4-8423-e689c5b9ca5b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.261134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44de87-028e-44f4-8423-e689c5b9ca5b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.261407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44de87-028e-44f4-8423-e689c5b9ca5b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.263727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 303/605] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:47:39 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/fbc41720695b45ec84c2f250238ac37b/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:47:40.268267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 303/606] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:40.269618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.269753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:40.269926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.270222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.273969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:40.305701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:40.307204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:40.460025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 304/607] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:40.482441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:40.487062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5336b09-7987-485e-84b7-bdacf7ace67a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.525678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.526355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.526642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.526923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.527760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:40.528073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:40.529505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.542451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 304/608] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:40.561574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:40.581564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:40.601544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f6c50-52b6-4125-abac-016ac1a62a81 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.649427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.650606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.651143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:40.652346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.652833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.680363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:40.695398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:40.696437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:40.697351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:40.697980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d5e6-6242-43ff-9f4a-e224326b7510 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.700349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 305/609] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:40.707127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:40.736183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.737011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.737295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.737574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/6aae965af8284363820254c9286adf96/roles/777a339245484c7fbe89538db336a995` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.770573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=6aae965af8284363820254c9286adf96, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:40.772059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.853340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 305/610] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:47:40 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/6aae965af8284363820254c9286adf96/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:47:40.864725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7609ba-5f5c-4b27-abe9-590b7a842818 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:40.906520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:40.907012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:40.907301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:40.907641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:40.908407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:40.908901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:40.910466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:40.920333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 306/611] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:40.927221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:41.061023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda220c8-1173-4fab-a046-8ee529baa304 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:41.082967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:41.105486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:41.106390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.106744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.107135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.119659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:41.121390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:41.142163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:41.144071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:41.145124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:41.145124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:41.145124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:41.145124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:41.145124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 306/612] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:41.156714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:41.159818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:41.291136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe234e-91b5-4c56-b152-3e9f72dc4497 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:41.340536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:41.341125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.341407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.341683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.342404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:41.342710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:41.344616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f3479-4a2e-46f8-b173-35ac6aa1ccee None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:41.357132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 307/613] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:41.363796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:41.496398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbb3fb-ee4b-4157-b18d-78103938e1f8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:41.543145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b04a6-631e-4d44-8322-4924c47aac0b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.543449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b04a6-631e-4d44-8322-4924c47aac0b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.543729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b04a6-631e-4d44-8322-4924c47aac0b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.545446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 308/614] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:41.552837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:41.553890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.554039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.554178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.554459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.554760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.555182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.572522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:41.573878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:41.574432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:41.575186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:41.600359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:41.601530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:41.602677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:41.604923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 307/615] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:41.610578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:41.647877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:41.677203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:41.698718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 309/616] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:41.737461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ddfed-693c-4e34-9387-3dee849a294f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:41.741098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:41.775631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:41.783395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.783395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.783395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.783395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:41.783395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:41.783395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:41.803455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 308/617] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jun 16 17:47:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:41.803455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:41.889742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:41.910124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d5efa-0eb1-487d-afdd-6f35e5862474 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:41.920093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac0fc0-76d1-47c0-bdae-b1cfaedc7758 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:41.947115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:41.948188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.948644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.949068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.960495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:41.962196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:41.980693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:41.987969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:41.989089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:41.989089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:41.989089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:41.989089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:41.989089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 309/618] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:41.989452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:41.989452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:41.989452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:41.989452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:41.991097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.009802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:42.018355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:42.019520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:42.020531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 310/619] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:42.026010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.086793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.167810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e245c8-8c7a-42bf-af31-fab815a23252 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.185284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d59d3f-8f85-4504-9479-3a21bf089247 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.206425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.207092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.207405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.207875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.209262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:42.209813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.211187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88c01-2190-4ecb-bc27-53118a5a68ad None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.221243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 311/620] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:42.226876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81787078-61d2-44c2-9d4e-224ac23f4a76 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.227168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81787078-61d2-44c2-9d4e-224ac23f4a76 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.227483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81787078-61d2-44c2-9d4e-224ac23f4a76 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.232841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 312/621] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:42.232841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81787078-61d2-44c2-9d4e-224ac23f4a76 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.235424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.236182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.236363 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.236640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.237431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:42.237735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.239152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8272a-7d4d-47a9-a5cb-921d7df6cbae None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.240720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.259128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 310/622] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:42.260462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.260747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.261026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.280201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:42.282747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:42.351515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81787078-61d2-44c2-9d4e-224ac23f4a76 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.371395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 311/623] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:42.371395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.374690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.413272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.414483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.414997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.415513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.429935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.431722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:42.449481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:42.450424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:42.451331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:42.451331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:42.451331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:42.451331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:42.454306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 313/624] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:42.455346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.486298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079b9dc-8ce5-4b65-92c2-d998bc05578e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.513303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.526841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.527724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.528001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.532653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.536588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.538038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:42.553875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:42.554889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:42.555811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:42.556724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 312/625] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:42.557238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.567646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7172212-020c-4b4c-abca-2d7ce5754cba None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.607224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.608463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.608987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.609516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.620587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:42.629370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:42.630289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:42.631129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:42.638361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 314/626] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:42.639470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.668838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.673853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889122ce-4e89-41df-b4db-5274a6f66e12 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.712318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.712909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.713263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.713457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.714173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:42.714474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.715972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.729521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 313/627] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:42.735215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.749817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b688918-4d39-4310-bc46-818d40be9c5f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.785529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.786026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.786798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.787314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.787927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.788963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:42.789492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.791224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.805066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 315/628] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:42.806261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.850051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b76e6-532e-47ac-a65b-46ac61725e4d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.895462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.895889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.896174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.896444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.897196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:42.897500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.899107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:42.913247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 314/629] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:42.913247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:42.922345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:42.938608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fb561-4dff-4fb7-8fe4-888f673b1899 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:42.974836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:42.975577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:42.975858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:42.976132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:42.985386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:42.987057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:43.003149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:43.004466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:47:43.005657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:43.008824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 316/630] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:43.009913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.025516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a7d7-4ef7-4c2c-8db6-d7de96a84196 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.033231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.061371 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301f01-f379-4c25-b60d-a20c13f2d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.063114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301f01-f379-4c25-b60d-a20c13f2d39e None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.063425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301f01-f379-4c25-b60d-a20c13f2d39e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.063709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301f01-f379-4c25-b60d-a20c13f2d39e None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/8bd57a7e4c074ca2890ec6bc081809cf/roles/777a339245484c7fbe89538db336a995` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.080411 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12301f01-f379-4c25-b60d-a20c13f2d39e None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=8bd57a7e4c074ca2890ec6bc081809cf, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.081861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12301f01-f379-4c25-b60d-a20c13f2d39e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.116769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4437-c5e7-4379-8204-f501699dc75d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.150289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.151140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.151464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.151751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.154140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 315/631] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:47:42 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/8bd57a7e4c074ca2890ec6bc081809cf/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:47:43.164751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.165032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94834ee0-a822-4a63-bc02-3ca2706932b1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.165415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94834ee0-a822-4a63-bc02-3ca2706932b1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.165767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94834ee0-a822-4a63-bc02-3ca2706932b1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.166223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.167495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 316/632] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:43.168135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94834ee0-a822-4a63-bc02-3ca2706932b1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.174912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-12ed6c9a-a931-4ef0-97b3-e66a140eb4a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:43.188346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:43.189330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:43.190278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:43.190278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:43.190278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:43.190278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:43.194420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 317/633] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:43.196160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.196337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.196733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.215313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:43.216835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:43.310086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 318/634] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:43.311052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.324489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94834ee0-a822-4a63-bc02-3ca2706932b1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.365431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.366215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.366503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.366878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.367754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:43.368068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.369495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.391471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 317/635] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:43.392153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.409514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.425178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f63b7c-bb0c-41a3-8111-fa95f0c7bdcd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.458248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.466213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.467054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.467437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.467841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.468958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:43.469348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.470789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.486804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 319/636] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:43.487689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.521280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e543d01-aaee-4b4c-8625-641ca263c53c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.560021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dddba-41cc-44ed-b938-d273f49edddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.560893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dddba-41cc-44ed-b938-d273f49edddb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.561251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dddba-41cc-44ed-b938-d273f49edddb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.561528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dddba-41cc-44ed-b938-d273f49edddb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.573123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dddba-41cc-44ed-b938-d273f49edddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-997dddba-41cc-44ed-b938-d273f49edddb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:43.582037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:43.582985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:43.584218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 318/637] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:43.584908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a6e41-2119-4de2-8dde-ed2872c6ae57 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.584908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a6e41-2119-4de2-8dde-ed2872c6ae57 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.585338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a6e41-2119-4de2-8dde-ed2872c6ae57 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.586541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 319/638] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:43.587097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a6e41-2119-4de2-8dde-ed2872c6ae57 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.604967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7643bb-18a1-4fc1-a43d-bdc30fe06326 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.640797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6e933-b58c-4c8a-bbb6-3be4e33bc795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.641477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e933-b58c-4c8a-bbb6-3be4e33bc795 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.641766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e933-b58c-4c8a-bbb6-3be4e33bc795 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.642048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e933-b58c-4c8a-bbb6-3be4e33bc795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.672963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.701647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a6e41-2119-4de2-8dde-ed2872c6ae57 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.702827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.718467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e933-b58c-4c8a-bbb6-3be4e33bc795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.719981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e933-b58c-4c8a-bbb6-3be4e33bc795 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.741032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.742164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.742681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.743228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.752324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.753893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:43.768384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:43.769541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 320/639] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:43.770564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.828494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 320/640] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:43.830065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.830496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.830937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.848669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:43.850041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:43.884211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bc6dc-7579-4baf-a963-c92e9d25b050 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:43.924061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 321/641] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:43.924906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:43.925071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:43.925881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:43.926288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:43.926687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:43.928595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:43.929074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:43.930559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:43.932182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.945941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:43.950581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 321/642] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:43.950581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.039702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85211b89-de4d-4caa-b24c-3cfb5fca0c2a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.062364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b28e0-568c-4053-963b-b48392740724 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.075834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.076511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.076802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.077083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.078193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:44.078569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.079985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.098968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 322/643] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:44.099212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.100015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.100302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.100580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.100899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.114473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.115950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:44.134477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:44.135644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:44.136870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:44.136870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:44.136870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:44.136870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:44.140283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 322/644] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:44.141204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.189587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.196076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.211881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a074f74-7c06-499c-b1b1-2df0fa03d322 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.245789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af93c0-ca87-4bd3-b137-81cbad576f4a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.251207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.254473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.254473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.254473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.266885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:44.274828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:44.275989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 323/645] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:44.276991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.286119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.286930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.287213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.287543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.299047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.300522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:44.318550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:44.319489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:44.325681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:44.325681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:44.325681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:44.325681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:44.325681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 323/646] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:44.325681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.395273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778bcad-05fb-42b1-a82e-fdda69515276 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.429615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.431942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.432515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb4b60-cb7f-451f-9802-9e4770da0798 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.432735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.433019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.433294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.437844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.469572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.470256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.470539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.470811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.471556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:44.471863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.473223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.489137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 324/647] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:44.489137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.507389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.508834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.592267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb292df-d9b7-4487-a3cf-9b2bcf26a7bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.630414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 324/648] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:44.631274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.636478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.637245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.637604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.637953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.638791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:44.639170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.640698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.649748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 325/649] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:44.650548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.673010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.679314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.740256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00258287-2f8b-405c-864f-d50b29a935d6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.755000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c538e1-d32c-40a9-aa36-f4c6ba15594e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.775962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.776625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.776909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.777186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.777899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:44.778206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.779588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.791155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.796969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 325/650] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:44.797125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.797125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.797125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.797387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.804652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.806172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:44.820665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:44.821578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:44.822458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 326/651] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:44.822941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.902112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.909084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a5136-0386-4279-91ca-febe28a10e50 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.916611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:44.937731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced01d2d-218f-464a-91ef-8780c3233b83 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:44.948087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.948887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.949180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.949441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.961336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.962795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:44.974779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:44.975608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:44.975897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:44.976176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:44.981604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:44.985841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:44.986988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:44.986988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:44.986988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:44.986988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:44.986988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 326/652] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:44.986988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:44.992709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:44.994187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:45.013977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:45.015189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:45.020900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:45.020900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:45.020900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:45.020900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:45.020900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 327/653] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:45.020900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.097209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72195b-4916-4847-a1af-2be4fc6a5fc5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.125376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ad96-f62e-4aa8-99c5-1cce1ed1ad5a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.133132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.134013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.134365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.134708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.143906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.145357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:45.160012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:45.161167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:47:45.162646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:45.164927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.165739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 327/654] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:45.165941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.166305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.166564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.166721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.167623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:45.168008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.169661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.171731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.177806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.188375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 328/655] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:45.188375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.277129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24fa62-a3fc-4695-80b0-932381821bf9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.306401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400bc35-e86b-49c7-9b25-7bbc0a5f0260 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.314321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.315138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.315556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.315942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.316766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:45.317174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.318621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.331568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 328/656] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:45.332492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.348764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.349508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.349789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.350115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.350899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:45.351208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.352653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.366958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 329/657] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:45.367870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.411930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.430841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.446044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadd73c-23e8-4fa7-ae9c-31b4dfd11791 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.475529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57e9d7-bc1f-436c-83bf-fc40e432de4f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.482178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.483250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.483637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.483917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.485112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:45.485431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.487160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.504275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 329/658] 10.222.0.58 () {56 vars in 1137 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:45.505075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.509459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.510189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.510472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.510750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.511642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=6f37017740874fef845f23fca4f977f4) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:45.511957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.513384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.535398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 330/659] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/role_assignments?user.id=6f37017740874fef845f23fca4f977f4&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:45.535398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.616445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1eaf-dae1-4b84-bcde-81dd215adc76 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.649251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.649860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b83f9c-dbf6-4d02-84a3-12616612aa81 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.659221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.660348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.660747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.661124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.668930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.669718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.671541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-425c91cd-d628-41c5-8173-ae7f8b248b75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:45.687204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:45.689296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:45.690426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:45.690953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.690953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.690953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.690953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.696512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 330/660] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:45.696512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6feb6b-1110-48da-bd25-50bc01569932 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.696512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6feb6b-1110-48da-bd25-50bc01569932 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.696512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6feb6b-1110-48da-bd25-50bc01569932 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.696946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 331/661] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:45.697748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6feb6b-1110-48da-bd25-50bc01569932 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.704321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.705857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:45.725103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:45.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:47:45.727538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:45.729011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 331/662] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:45.729897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:45.816871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6feb6b-1110-48da-bd25-50bc01569932 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.836230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f99af-635c-4d81-b652-0ad4635410d1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:45.853439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.854097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.854431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.854708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.871549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:45.872221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.872509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.872785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.873569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:45.874044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.876165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df405b-1478-4492-8440-4e83e10782f9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.893465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 332/663] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:45.893465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:45.893465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:45.893465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:45.910127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:45.911325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:45.916288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.930273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:45.931727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:45.948824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:45.993893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 333/664] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:45.993893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.056214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 332/665] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:46.057281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.107200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148fef1-500b-403d-940e-2a3ac8aef1bf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.144445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.145132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.145420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.145700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.146563 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:46.146856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.148315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.155180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:46.173532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 334/666] 10.222.0.58 () {56 vars in 1139 bytes} [Thu Jun 16 17:47:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:46.173532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.177799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31142ce-996f-4fed-beba-eb506dde9a6e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.202309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:46.220801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.221461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.221733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.222008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.222886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=9279195b1ed64901a63a3fce8d0bb610) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:46.223194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.224717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.239484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 333/667] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/role_assignments?user.id=9279195b1ed64901a63a3fce8d0bb610&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:46.239484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.300934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41ab34-1047-474b-9d88-cdcea37fdbb6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.349330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.351219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.352018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.352456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.368185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:46.376566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802454a-ea99-47d0-9939-5868af1f6e14 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:46.380335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:46.381526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:46.382566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:46.385112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 335/668] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:46.386300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.427950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.428161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.428417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.428620 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.440720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.442245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:46.467590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:46.468957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:46.469842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:46.469842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:46.469842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:46.469842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:46.472097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 334/669] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:46.473095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.479355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:46.498789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:46.520893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19f2a7-a38c-460e-9e59-d899735848b5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.559870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.560656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.562659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.562947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.611877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d063a4-0751-4eda-b9cc-9a3771a6217e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.637667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.639170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.656886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.658381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.659169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.659676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.660891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:46.661787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.663357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.678161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 335/670] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:46.679625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.737773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:46.757197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 336/671] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:46.758060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.771029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:46.817403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf292e-049e-4b7e-92ac-7886682c63a5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.864588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.865262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.865547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.865824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.866547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:46.866854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.868248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.876813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd90-bf16-419e-b86b-190346d52a82 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:46.880852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 336/672] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:46.881679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:46.919099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:46.920192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:46.920479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:46.921207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:46.934833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:46.936782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:46.969117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:46.970144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:46.971019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:46.971019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:46.971019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:46.971019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:46.975087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 337/673] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:46.975972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.000083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf7f1a-2976-4bb3-a0d1-9c183c229f5e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.020385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.026088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.037691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.038640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.038928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.039205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.052466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.054323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:47.077065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:47.083723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:47.084654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:47.084654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:47.084654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:47.084654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:47.084654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 337/674] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:47.084654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.099646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e9aaf-9e11-4a21-b256-8c2aa9d75912 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.137885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.139109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.140443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.141243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.149156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.150991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:47.167952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:47.169508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:47.170390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:47.178450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 338/675] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:47.179677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.204623 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fdbb4-e5c5-435e-8a5b-91cc0c1e7269 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.254016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.254613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.254898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.255167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.255922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:47.256223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.257582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.275332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 338/676] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:47.276531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.309900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.315145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5d53-e1ad-4201-bf59-88e343d747ef None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.315275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.358838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.360932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.361376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.361838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.362726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:47.363182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.364717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.381472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 339/677] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:47.382628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.398859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22da25-bda2-48ae-9622-f143337a855c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.436144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.437725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.438044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.438394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.439312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:47.439734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.441360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.459875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 339/678] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:47.460830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.510574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39785861-b7fb-4ea9-a295-3567c7b7d09f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.547684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.547903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.548437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.548629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.557649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.558992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:47.577353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:47.578384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:47.579323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:47.583245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 340/679] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:47.584161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.585984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.594998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6019976-843d-4e6c-bb24-fdb1da35becb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.612296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.650585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.651764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.652763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.653244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.663929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.666347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:47.689264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:47.690217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:47.691143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:47.697011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 340/680] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:47.697982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.712237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209efce-30b1-484d-904b-ff5715b767e4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.764329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.766332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.766797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.767521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.768806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=fbc41720695b45ec84c2f250238ac37b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:47.769746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.771954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.781049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 341/681] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/role_assignments?user.id=fbc41720695b45ec84c2f250238ac37b&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:47.782226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.833281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0792d-839f-49db-a549-081cb6d01f93 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.872561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.872926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.873147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.874674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:47.875029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.876687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43b2fb-6183-4a4a-bc35-91905f52ea66 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.902013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 341/682] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:47.904413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0324-52ce-4f57-aa13-a7880c2d3477 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.904886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0324-52ce-4f57-aa13-a7880c2d3477 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.905365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0324-52ce-4f57-aa13-a7880c2d3477 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.907035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 342/683] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:47.908370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.913454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0324-52ce-4f57-aa13-a7880c2d3477 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:47.932287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:47.935823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97865889-adb4-457b-b9a1-9ab0e238ba4c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:47.980083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:47.981085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:47.981368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:47.981642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:47.982432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:47.982758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:47.984903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbc512-78bf-48e1-a1a6-b3dd71ef95fc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:47.997470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 342/684] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:47.999652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.000136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.000434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.020770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:48.021916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:48.050247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0324-52ce-4f57-aa13-a7880c2d3477 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.091245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.092003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.092219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.092527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.105736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 343/685] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:48.107251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.167846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.169431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.185681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:48.214375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:48.240979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65320-7d9e-4582-8443-90e2e791f430 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.291778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.293014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.293502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.293998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.295036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=6aae965af8284363820254c9286adf96) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:48.295587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.298673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.299692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 343/686] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:48.300601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.307473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 344/687] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/role_assignments?user.id=6aae965af8284363820254c9286adf96&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:48.308713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.433392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93445593-1ec8-4e88-accf-b21794dc4726 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.477351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.478604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.479092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.479586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.487768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:48.497083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.500304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:48.520653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:48.521611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:48.522535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:48.522535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:48.522535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:48.522535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:48.522535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 345/688] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:48.531284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e72ec-70bc-4d12-94af-3062229ef92f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.533157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.549131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:48.572506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.573303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.573589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.578584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.586783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.588268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:48.612081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:48.613069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:48.613966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:48.613966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:48.613966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:48.613966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:48.614640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 344/689] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:48.622163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.681021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a3cdb1-5dc9-47b2-b33f-ff02814240cd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.722493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.723256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.723643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.723996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.724918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:48.725296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.726802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfe4d0-2c1d-4569-9f20-45bec6e12ffe None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.741465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 346/690] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:48.744093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288c641-0d04-45e5-a7dd-60854df6bb01 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.745201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288c641-0d04-45e5-a7dd-60854df6bb01 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.745684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288c641-0d04-45e5-a7dd-60854df6bb01 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.747623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 347/691] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:48.748562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288c641-0d04-45e5-a7dd-60854df6bb01 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.756588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09645b10-ca17-4ffb-abde-275db6e6db4b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.810002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.811225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.812413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.812887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.814087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:48.814636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.815238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:48.817805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced56bc5-a2ec-45a6-8d98-29bec88f99c4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.835149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 345/692] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:48.837109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.838377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.838857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.861772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:48.862981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:48.879782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288c641-0d04-45e5-a7dd-60854df6bb01 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:48.885888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:48.928340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:48.929748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:48.930666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:48.931445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:48.943414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 346/693] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:48.943414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:48.947514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:48.949502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:48.970966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:48.971971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:48.972916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 348/694] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:48.973403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:49.086806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecffeb-7a61-405f-aad9-a5b56146ef43 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.104672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b5fee-e9e2-45c9-9bc0-6c014b5ac196 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.117603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:49.138450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.139134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.139548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.139845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.144381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:49.144381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:49.147699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.149012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.149546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.150449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.164151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:49.173661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:49.174598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:49.175463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:49.181715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 349/695] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:49.181715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:49.207494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 347/696] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:47:48 2022] POST /identity/v3/users => generated 254 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:49.208560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:49.245250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:49.326734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f849c50-7ee3-4bfb-b1bf-927cc776bd96 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.411229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.411857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.412140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.412419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.413217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:49.413528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:49.415252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:49.440663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf89ba-9ac4-4708-96ec-261e24e5bcfc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.441187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 350/697] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:49.447008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:49.477181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.477848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.478129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.478401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.479117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:49.479652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:49.479965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:49.481434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:49.493961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 348/698] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:49.500495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:49.578936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7768f-6f7a-45a7-9a9a-f95fdca01c10 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.597688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:49.625112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de0249-03be-401e-92df-7a476459a204 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.634390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.635128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.635445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.635729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.636995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=8bd57a7e4c074ca2890ec6bc081809cf) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:49.637314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:49.639302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:49.647848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 351/699] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:49 2022] GET /identity/v3/role_assignments?user.id=8bd57a7e4c074ca2890ec6bc081809cf&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:49.669582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.671049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.671602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.671960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.687197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:49.688839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:49.708414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:49.709418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:49.711354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:49.711354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:49.711354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:49.711354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:49.711354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 349/700] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:49.716071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:49.853360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:49.862397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c3361-d62d-43d5-ab4f-feaefc4d56e8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:49.898373 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:49.899054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:49.899345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:49.899654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:49.900396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:49.900712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:49.902266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:49.914948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 352/701] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:49.920313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:50.039554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b443a01-dfec-4da4-a0c2-7af2b325c16c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:50.067989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:50.077773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:50.078706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.079080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.079395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.087337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:50.088851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:50.103912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:50.104901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:50.106466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 350/702] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:47:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:50.111818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:50.218408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412c6c6-7ea2-40a6-a4b9-ff5d776284bd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:50.257822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:50.257822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.258109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.258338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.259406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:50.259406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:50.260848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450091fb-90d4-42fa-b325-5846b364414b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:50.280298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 353/703] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:47:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:50.286083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:50.404004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad2b31-cb35-4d23-8881-bd620b07d7d4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:50.452989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:50.453655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.453942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.454218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.496979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642fc6f-a488-4c3e-b361-7cddb191622a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.497941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642fc6f-a488-4c3e-b361-7cddb191622a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.498482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642fc6f-a488-4c3e-b361-7cddb191622a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.500446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 354/704] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:50.506825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.507416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.508074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.547348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:50.550438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:50.550961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:50.559707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:50.580572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:50.582042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:50.721938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 355/705] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:50.749883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 351/706] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:50.755392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:50.768394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:50.873389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:50.895553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fb3ab-7130-4453-917b-89cc51ef9ff8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:50.933648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44450de9-a33e-4131-ab55-778446058059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:50.934772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7489e78-ab98-476d-90a0-583e028d85e6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:50.935077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44450de9-a33e-4131-ab55-778446058059 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.935622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44450de9-a33e-4131-ab55-778446058059 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.936713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44450de9-a33e-4131-ab55-778446058059 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.937669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44450de9-a33e-4131-ab55-778446058059 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:50.939246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44450de9-a33e-4131-ab55-778446058059 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-44450de9-a33e-4131-ab55-778446058059 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:50.954961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:50.956035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:47:50.956900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:50.960086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 352/707] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Jun 16 17:47:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:50.961685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffa789-4fb8-42fa-b885-34909213eb48 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.962181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffa789-4fb8-42fa-b885-34909213eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.962688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffa789-4fb8-42fa-b885-34909213eb48 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.968569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 353/708] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:50.970839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.971836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.972469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.978499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:50.979348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:50.979940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:50.980225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:50.989621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:50.992052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:50.993633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:50.995691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:50.997135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:51.013666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:51.015198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:51.016871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:51.016871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:51.016871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:51.016871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:51.020161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 356/709] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:51.021098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.083494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 354/710] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:51.084818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.129660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de70007-103f-4e5f-b6e5-b27570c2f5ba None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.176187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.176870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.177160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.177441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.178536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:51.178851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.180901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.188071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 357/711] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:51.188913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.200747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:51.220989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4b164-1b86-439a-ad51-ba725c6ed35b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.277438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.278117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.278405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.279419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.279548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:51.283580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:51.285681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.285681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.303455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 355/712] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:51.304661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.345903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf3ad0-a959-4472-8fa2-c45fe0c247d9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.383095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.384284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.384779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.385285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] PATH_INFO: `/v3/services/network` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.386137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.387750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:51.402475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:51.403574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:47:51.404506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:51.411953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 358/713] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:51.413050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.431604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0c391-6d0b-4ba4-96fb-7c64d2094bd9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.480306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.481123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.481893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.482202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.483779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:51.484125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.485718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.499474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 356/714] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:51.500868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.509268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:51.568470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890549-f4c7-4702-ab00-0c31d0f2347e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.620452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:51.643903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c5e44-261e-4b7a-90d8-a6556c3e74c3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.678564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.679480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.679830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.680120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.690411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fc098-9548-4339-9661-be96869aed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.691529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fc098-9548-4339-9661-be96869aed4e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.692027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fc098-9548-4339-9661-be96869aed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.692544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fc098-9548-4339-9661-be96869aed4e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.693468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fc098-9548-4339-9661-be96869aed4e None admin] RBAC: Adding query filter params (name=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:51.693984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fc098-9548-4339-9661-be96869aed4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.695800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fc098-9548-4339-9661-be96869aed4e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.700695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.702666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 357/715] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:51.702873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.709217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fc098-9548-4339-9661-be96869aed4e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:51.727403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:51.728369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:51.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 359/716] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:51.730976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.825962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:51.872746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fc098-9548-4339-9661-be96869aed4e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.878130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27d048-eef9-4fda-891e-e3dcefe4f723 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:51.917255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.918239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.918700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.919137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:51.919402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.919966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.920414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.921001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.921399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:51.921927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.922036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:51.922331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:51.924093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e58db-fa2c-48d7-9f8b-f0055b86be12 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.924221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:51.930483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 358/717] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:51.932063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc30aa-ca98-4d9c-9d6a-fc95b8d53adb None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.932522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc30aa-ca98-4d9c-9d6a-fc95b8d53adb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.932968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc30aa-ca98-4d9c-9d6a-fc95b8d53adb None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.934732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 359/718] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:51.941467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:51.942129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:51.942594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:51.953830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 360/719] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:51.959782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:51.962898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:51.964414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:52.054435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:52.121357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 360/720] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:47:52.140907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:52.147041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3fe82-c3db-4b46-ad43-782ae2ad28ae None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:52.215221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:52.216580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:52.217060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:52.218124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:52.220006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:52.220558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=9279195b1ed64901a63a3fce8d0bb610) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:52.221117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:52.222773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:52.230840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 361/721] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:51 2022] GET /identity/v3/role_assignments?user.id=9279195b1ed64901a63a3fce8d0bb610&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:52.255391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed85bc-9461-42f8-9375-d11f583aa8df None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:52.295933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:52.296765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:52.297051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:52.297329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:52.310298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:52.319071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:52.320126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:52.321225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 361/722] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:52.329083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:52.495569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6a3b5-42a5-4023-a47f-6ce51b240581 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:52.504458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:52.549370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:52.550391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:52.550888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:52.551343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:52.552327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:52.552825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:52.554399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425b1de-a074-47c9-9b05-4a9dd5a1a954 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:52.568071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 362/723] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:52.573943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:52.640989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f080b-d627-4b79-959f-c527d6a648e8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:52.641904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f080b-d627-4b79-959f-c527d6a648e8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:52.642363 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f080b-d627-4b79-959f-c527d6a648e8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:52.643976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 363/724] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:52.650684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:52.651318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:52.651813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:52.687285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:52.688933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:52.738077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29095d-89ed-456b-8416-71cf3ce79fe1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:52.783814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:52.788632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:52.789439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:52.790121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:52.790121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:52.803473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:52.811153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:52.825536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 364/725] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:52.834702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:52.836031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:52.837279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:52.837279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:52.837279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:52.837279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:52.837279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 362/726] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:52.843865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:52.901651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.032466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:53.041539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f949e48-a710-48cf-b96e-68fb4fa07f69 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.093395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd50468-5504-406b-b0d1-b2a3b89c0953 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.134756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.136710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.137597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.138495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.143381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.145129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.149815 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.150399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.151393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:53.151917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:53.153508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:53.174476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 365/727] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:53.179601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.184925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:53.193926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:53.195050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:53.195955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:53.199131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 363/728] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:53.206436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.258265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:53.336420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693510d6-2a49-4c46-9da7-3ee5471018f7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.379147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.380186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.380942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.381436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.382532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:53.383036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:53.384624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:53.396725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 364/729] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:53.402606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.469246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb7ab-948b-4800-af0e-2ca9a5f95626 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.516111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:53.524519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea25f-11e2-4d2a-95be-339933cd3226 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.565701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.567062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.567586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.568067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.574463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.575296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.575605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.575881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.583068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:53.584823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:53.596191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:53.597569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:53.606274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:53.607225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:53.609465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:53.609465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:53.609465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:53.609465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:53.612005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 365/730] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:53.617804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:53.634185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:53.635200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:53.636198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 366/731] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:53.641284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.679716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:53.777997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f10d4c-1321-4dad-b7a5-483ead2df489 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.812253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47c95d-f89c-49e0-8fda-493e4e4c6d58 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:53.856168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.856817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.857048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.857324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.858866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:53.858985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:53.860394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:53.872851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 366/732] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:53.874618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:53.875311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:53.875619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:53.875896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:53.876736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:53.877045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:53.879668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:53.882468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:53.890839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:53.921644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 367/733] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:53.927448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.022176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dc03c-9542-4736-9c15-afccfa67ad43 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.026644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.054630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04388f-1d3f-4552-9f37-37ef1223785f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.059776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.060894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.061333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.061915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.069734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.071528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-869939ba-2aa9-44bb-83f1-28bff0a0e1eb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:54.088737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:54.089911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:47:54.091075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:54.094291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 367/734] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:54.094411 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.095194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.095619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.095977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.096415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cced5-d3a2-4ece-8756-b97e2f05a3df None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.096978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cced5-d3a2-4ece-8756-b97e2f05a3df None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.097436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cced5-d3a2-4ece-8756-b97e2f05a3df None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.103548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 368/735] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:54.104570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cced5-d3a2-4ece-8756-b97e2f05a3df None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.169510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.171667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.180771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.218508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cced5-d3a2-4ece-8756-b97e2f05a3df None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.254482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.258884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.259881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.260369 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.260840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.261863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:54.262388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.264104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0888e-b0c6-4b77-9b65-b464b8f73f4a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.283227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 369/737] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jun 16 17:47:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:54.284990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.285488 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.285987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.286111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 368/737] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:54.287723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.305302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:54.306938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:54.387745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 370/738] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:54.389467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.394789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b73008-432f-4545-9e99-4b3bb17333f6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.431763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.432601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.432872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.433154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.446151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.447738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:54.466437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:54.468166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:54.469092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:54.469092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:54.469092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:54.469092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:54.473198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 369/739] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:54.474071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.477323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.505409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.525619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb65c6-c34c-40c6-830a-a05e28cb8cf3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.570606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe659742-8238-40e7-969a-cab8b8c381dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.571289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe659742-8238-40e7-969a-cab8b8c381dc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.571599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe659742-8238-40e7-969a-cab8b8c381dc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.572715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe659742-8238-40e7-969a-cab8b8c381dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.586482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fa2b9-47d8-4554-953d-e3d4f321b39f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.626755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.627466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.627753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.628028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.629312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.630682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.660314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe659742-8238-40e7-969a-cab8b8c381dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.661734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe659742-8238-40e7-969a-cab8b8c381dc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.723848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.728134 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 370/740] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:47:54 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:54.729307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.816546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 371/741] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:47:54.818662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707236f-a56a-4008-adf7-63d61bbb6fce None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.819541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707236f-a56a-4008-adf7-63d61bbb6fce None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.820571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707236f-a56a-4008-adf7-63d61bbb6fce None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.822990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 372/742] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:54.825035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707236f-a56a-4008-adf7-63d61bbb6fce None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:54.854711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:54.878574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3974fed-6aa6-4d30-af6c-aeb1db3eca55 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:54.918399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:54.919087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.919569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.919860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.921017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:54.921017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:54.922298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d07a3-5f39-42aa-8059-1ba3fd358e69 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:54.935936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 371/743] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:54.937438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc12a42-8ceb-439d-8bb4-29fb55c1854a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:54.937728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc12a42-8ceb-439d-8bb4-29fb55c1854a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:54.938005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc12a42-8ceb-439d-8bb4-29fb55c1854a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:54.955943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2cc12a42-8ceb-439d-8bb4-29fb55c1854a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:54.957156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2cc12a42-8ceb-439d-8bb4-29fb55c1854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:54.962813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707236f-a56a-4008-adf7-63d61bbb6fce None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.002111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.002881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.003166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.003467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.019318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.020870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.036044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 372/744] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:55.037805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974176b-7b12-498a-b445-1c8e2a3352a3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.038309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974176b-7b12-498a-b445-1c8e2a3352a3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.038798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974176b-7b12-498a-b445-1c8e2a3352a3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:55.040199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:55.041069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:55.041972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:55.041972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:55.041972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:55.041972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:55.046857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 373/745] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:55.047345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 373/746] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:55.047846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.048954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974176b-7b12-498a-b445-1c8e2a3352a3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.059830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.130525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.159251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707e942-7575-46e2-8316-d6b4c7943060 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.163312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974176b-7b12-498a-b445-1c8e2a3352a3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.196173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.197133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.197414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.197609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.200734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.201933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.202216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.202492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.203210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:55.203545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.204918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c01b0-cedf-43f4-a4a0-7ed817c5e8f7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.205366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.206791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.221861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 374/747] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:55.221861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.221861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.221861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:55.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:55.223200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 374/748] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:55.224127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.240720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:55.245237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:55.302921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.319719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 375/749] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:47:55.324296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.343938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97070c-9b50-430f-99bd-5315f19ff671 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.380763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.381563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.381845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.382121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.393312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:55.397394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:55.400830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:55.401767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:55.402736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 375/750] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:55.403461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.446219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9237973-b287-482a-ae7d-5c082e7cba25 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.482983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.484166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.484639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.485183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.492944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.494587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:55.509297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:55.510262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:55.511160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:55.511627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfec581-bcca-4430-936a-a3b6d6e90c48 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.515526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 376/751] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:55.516539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.538860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.547194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.548623 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.549029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.549314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.550268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:55.555037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.555037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.569073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 376/752] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:55.570092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.628375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e562-b5ff-45b4-8076-b93abee60c9c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.652083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.663689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.664360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.664644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.664924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.669393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.669393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.679065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e6803-ad78-4e43-a55e-cb8f9d9b60e2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.712619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.713512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.713801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.714119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.714858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:55.715169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.716581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.729265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 377/753] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:55.729265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.734970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 377/754] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:47:55 2022] POST /identity/v3/services => generated 240 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:47:55.736654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.794832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:55.853764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011470e2-e6c4-4a5a-9868-85e188a6df72 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.894567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f0b1-fcf6-4001-a0ce-2e50446f92bf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:55.906145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.909301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.909477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.909719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.910602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:55.912092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.913815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.932953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 378/755] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:55.933405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:55.943978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:55.944648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:55.944935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:55.945211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:55.946317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=df08ee888dfd4aeb8025b594fcb2702e) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:55.946504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:55.947935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:55.955178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 378/756] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/role_assignments?user.id=df08ee888dfd4aeb8025b594fcb2702e&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:55.956059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:56.016657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:56.064731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc601e-64c1-459d-971b-13be3adfa70c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:56.073244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:56.097156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407139f-b029-43f8-ad6c-3ab34856fc2d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:56.102810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:56.104130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:56.104645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:56.105129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:56.118438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:56.120344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:56.155433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:56.157023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:56.158346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:56.158346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:56.158346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:56.158346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:56.158346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 379/757] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:56.158632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:56.158632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:56.158632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:56.158632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:56.159447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=8bd57a7e4c074ca2890ec6bc081809cf) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:56.159619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:56.161434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:56.162667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:56.185661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 379/758] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:47:55 2022] GET /identity/v3/role_assignments?user.id=8bd57a7e4c074ca2890ec6bc081809cf&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:56.290587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60172-ebe7-41b7-9f27-309804c7b57f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:56.331379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:56.332078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:56.332366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:56.332639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:56.333376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:56.333681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:56.335057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:56.349376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 380/759] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:56.357354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:56.510038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a885d-3ce8-4ab7-aef2-00bf4a4e469c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:56.531780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:56.546913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:56.548134 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:56.548521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:56.548895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:56.557081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:56.558814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:47:56.575835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:56.576900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:56.577851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 380/760] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:47:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:56.588472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:56.619782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:56.740175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f9af3-71e8-42e1-aafc-b824c75d3704 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:56.791997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:56.792672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:56.792956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:56.793231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:56.794077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:56.794387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:56.796390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:56.826850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 381/761] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:47:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:56.832406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:56.995719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96e5b5-66ea-4e4e-81ce-334a8ebd0548 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:57.049965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:57.050775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.051056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.051337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.071008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:57.072484 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-84d79ae3-9e1b-409e-8990-3b1b209dc797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:57.094460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:57.095455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:57.096407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:57.096407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:57.096407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:57.096407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:57.099467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 381/762] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:57.104467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:57.193704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dce41b-612a-4740-985c-f308d8a85960 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.194354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dce41b-612a-4740-985c-f308d8a85960 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.194920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dce41b-612a-4740-985c-f308d8a85960 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.196987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 382/763] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:47:57.202375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e640-58c7-468b-9449-68b2db245529 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.202897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e640-58c7-468b-9449-68b2db245529 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.203414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003e640-58c7-468b-9449-68b2db245529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.230242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d003e640-58c7-468b-9449-68b2db245529 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:57.237697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d003e640-58c7-468b-9449-68b2db245529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:57.259856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:57.272803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:57.276200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62762887-1820-476c-a21c-9a78ca2f6580 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:57.317876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:57.318549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.318835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.319114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.319873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:57.320196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:57.322943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:57.346197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 382/764] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:57.353425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:57.404329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 383/765] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:47:57.426964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e640-58c7-468b-9449-68b2db245529 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:57.566934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd426a-ef8b-42bc-ae11-5487d6bccdc6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:57.611640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003e640-58c7-468b-9449-68b2db245529 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:57.630045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:57.630847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.631342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.631648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.637836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:57.644669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:57.646113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:47:57.669302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:57.670845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:57.671772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 383/766] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:57.683384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:57.687057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:57.689076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.689364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.689648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.691964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:57.693910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:57.716066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:57.717557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:47:57.718466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:57.723097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 384/767] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Jun 16 17:47:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:57.732007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:57.763807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:57.844770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901e7d9-6c6e-4bbc-b143-1a68c521c1fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:57.874771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48017c-1a30-4622-be45-614c25ce0e12 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:57.907286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:57.908105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.908514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.908800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.909592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:57.909901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:57.912056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:57.923296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:57.924186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:57.924621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:57.925070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:57.925923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:57.926376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:57.927912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:57.935720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 384/768] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:47:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:57.938023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 385/769] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jun 16 17:47:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:57.944136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:57.950349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:58.056292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:58.106738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:58.145985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056843f-c9af-4c18-b54d-5ce818369a26 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:58.152319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be502c36-3fd5-4fc3-85b2-d235512ca42a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:58.213221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:58.213888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.214172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.214449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.215154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:58.215483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:58.217397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:58.221179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:58.222075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.222356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.222629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/9279195b1ed64901a63a3fce8d0bb610/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.226469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=9279195b1ed64901a63a3fce8d0bb610, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:58.227957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:58.233121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 385/770] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jun 16 17:47:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:58.239393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:58.309843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 386/771] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:47:57 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/9279195b1ed64901a63a3fce8d0bb610/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:47:58.360925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a916b-4030-4393-8a5b-7dd1413dd484 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:58.382726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:58.402897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:58.403619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.403898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.404729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.406213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] RBAC: Adding query filter params (interface=public, service_id=4b46ae31fd354032827f840eb4143b53, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:58.406533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:58.407969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd07045-ba17-4ca5-90a2-d9b7fdcb8648 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:58.424165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 386/772] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:47:58 2022] GET /identity/v3/endpoints?service_id=4b46ae31fd354032827f840eb4143b53&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:58.429732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:58.564837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e2748-cbed-44ad-8730-5c01e2d00127 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.565191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e2748-cbed-44ad-8730-5c01e2d00127 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.565394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e2748-cbed-44ad-8730-5c01e2d00127 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.566937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 387/773] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:58.579744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39386d-ac6d-44f5-92ff-e42f38f43613 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.580043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39386d-ac6d-44f5-92ff-e42f38f43613 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.580323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39386d-ac6d-44f5-92ff-e42f38f43613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.611502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8f39386d-ac6d-44f5-92ff-e42f38f43613 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:58.612635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f39386d-ac6d-44f5-92ff-e42f38f43613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:58.642663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1c61e-02e1-47cd-8c13-02bf99154661 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:58.687214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ae15c-caee-4379-a3e5-07c80e8337e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:58.687972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae15c-caee-4379-a3e5-07c80e8337e6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.688260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae15c-caee-4379-a3e5-07c80e8337e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.688539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae15c-caee-4379-a3e5-07c80e8337e6 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.689223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ae15c-caee-4379-a3e5-07c80e8337e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:58.690640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ae15c-caee-4379-a3e5-07c80e8337e6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:58.692899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:58.693914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 388/774] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:47:58.695830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341ab7d-2d8e-4f7d-8f12-9317dfd2931c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.696338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341ab7d-2d8e-4f7d-8f12-9317dfd2931c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.696823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341ab7d-2d8e-4f7d-8f12-9317dfd2931c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.697243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 387/775] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:47:58 2022] GET /identity/v3/services => generated 772 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:58.698763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 389/776] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:58.705584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360068-b159-4c2d-926a-3be2f4b19565 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.706017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360068-b159-4c2d-926a-3be2f4b19565 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.706402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27360068-b159-4c2d-926a-3be2f4b19565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.726025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-27360068-b159-4c2d-926a-3be2f4b19565 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:58.727532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-27360068-b159-4c2d-926a-3be2f4b19565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:58.740691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341ab7d-2d8e-4f7d-8f12-9317dfd2931c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:58.841528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 388/777] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:47:58.843672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d5e52-dda6-4304-b1f1-06ea17599c50 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.843970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d5e52-dda6-4304-b1f1-06ea17599c50 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.844251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d5e52-dda6-4304-b1f1-06ea17599c50 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.850045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 389/778] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:47:58.851984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.852356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.852712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.884851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:47:58.888174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:47:58.924751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341ab7d-2d8e-4f7d-8f12-9317dfd2931c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:58.945921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:58.973950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:58.974896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:58.975182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:58.975492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:58.975743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 390/779] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:47:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:47:58.976576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:58.988233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:58.996827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:58.997792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:47:58.999189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 390/780] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:47:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:47:59.005143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.051066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.097316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915d76-0b78-4609-86cb-57992fa35da0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.122356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a54274-9c00-4fe1-ac44-13f2e37dce3c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.135152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.136227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.136656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.137170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] PATH_INFO: `/v3/services/network` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.137964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.141010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:59.153351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:59.155592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:47:59.156889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:59.160515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 391/781] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:47:58 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:59.160635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.161305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.161623 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.161901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.162656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:59.162964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.163717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.169366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.183518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 391/782] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:47:59.183518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.231474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.300786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ecc11-047a-4934-afc0-a9c3ebe8627f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.324895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.330512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c45c-5149-41c1-9b81-e5206bc6db34 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.342861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.344392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.344863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.345325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.361763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:47:59.369220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.369952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.370304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.370647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.371436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] RBAC: Adding query filter params (name=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:59.371829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:47:59.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:59.373304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:47:59.374260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:59.374935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.377324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 392/783] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:59.378559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.384039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 392/784] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:59.384039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.487544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.490637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be834cbf-c6cc-41b9-ad4c-00ce66569d54 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.515450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1b816-a77e-41cd-8153-df7dc9afeca4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.529146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.530477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.530914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.531157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.532006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:59.532395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.533852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.542953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 393/785] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:59.543984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.555925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.557033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.557563 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.558034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.570627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.572581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:59.593605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:59.594574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:59.595509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:59.595509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:59.595509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:59.595509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:59.597991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 393/786] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:59.599003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.637842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.659817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d979-c4b1-42e2-a89d-c0e98c7d8993 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.696326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.697089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.697471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.698007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.698783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:59.699159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.700741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.706368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 394/787] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:59.707242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.710697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0331f-3a0a-44dc-95ef-85ea26c98e7a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.736506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.750951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.750951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.750951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.750951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.762166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.764537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:47:59.783651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:47:59.784567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:47:59.785481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:47:59.785481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:47:59.785481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:47:59.785481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:47:59.787799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 394/788] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:47:59.788884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.813469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5747032-0187-4b25-9e03-bf73535245a0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.848707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.849297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.849586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.849866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.850705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:59.850885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.852275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.869044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 395/789] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:59.869934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.895772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.903894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a379b89-c4db-42e3-88a1-9a64d5d15132 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:47:59.950637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:47:59.952971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:47:59.953472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:47:59.953973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:47:59.955480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] RBAC: Adding query filter params (interface=public, service_id=ad7237e0d7ba4ca7820219c6ea3e95e1, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:47:59.956388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:47:59.958259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:47:59.963255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:47:59.970601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 395/790] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/endpoints?service_id=ad7237e0d7ba4ca7820219c6ea3e95e1&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:47:59.974247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:47:59.980929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9403-028c-4dc1-a439-ce896d843e95 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.022191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.023248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.023767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.024322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.025251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:00.026061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.027735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.040897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 396/791] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:00.042138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.105884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb2435-d2cc-4d18-9ea8-aedddfeac691 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.145298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.146475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.146978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.147509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.158616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.160837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.161796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f32c8-2b30-40d6-9445-ad9b08f2eb13 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:00.181576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:00.182570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:00.183555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:00.184821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 396/792] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:47:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:00.190026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.209500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.210577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:00.210693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.211398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.211587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.212330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.214103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.220482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 397/793] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:00.221303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.239012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:00.377107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e22e9d-e7ee-4c8b-8618-de6ed35f9d01 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.385161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e08919-2cb2-4c58-8b02-9fa9c726dec9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.425543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.430070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.430692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.431188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.432464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:00.433124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.437703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.472360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.473564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.474077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.474573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.482256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 398/794] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:00.489525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.489779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.491528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:00.510858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:00.511959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:48:00.512989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:00.516704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 397/795] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:00.522902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.596466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:00.622562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:00.632343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e321cc5-9f8a-4405-ba46-243a61173a90 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.666323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91b9f-8674-4ed9-8959-e728ddf8b9e1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.667967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.668767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.669048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.669324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.684470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.688530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.707324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.708386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.708888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.709393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.710455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:00.710979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:00.711304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:00.712214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:00.713187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:00.713187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:00.713187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:00.713187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:00.713187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 399/796] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:00.715269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.719026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.738041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 398/797] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:00.743605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.892943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097cdb2-3e35-48b5-aa5c-2e23dbd080f4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.915417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:00.928944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92b009-b91b-48d3-a787-da92d72b7776 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:00.934330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.934330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.934330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.934330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.934858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:00.934858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:00.936581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272d2f0-b3e6-4c15-8467-0c1529c949bc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:00.949656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 400/798] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:00.952610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7f863-dd5c-4c11-9217-26d77d0c6e8c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.953288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7f863-dd5c-4c11-9217-26d77d0c6e8c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.953960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7f863-dd5c-4c11-9217-26d77d0c6e8c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:00.956008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 401/799] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:00.957241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7f863-dd5c-4c11-9217-26d77d0c6e8c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:00.976254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:00.985942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:00.991845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:00.991845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:00.991845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.003772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.005248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d8f38b04-6b9a-4647-8718-df6c2b6d51d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:01.030936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:01.032063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:01.033091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:01.033091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:01.033091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:01.033091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:01.033091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 399/800] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:01.033830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640f24-0daf-4a51-a22e-097de2e43a6f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.034120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640f24-0daf-4a51-a22e-097de2e43a6f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.034399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47640f24-0daf-4a51-a22e-097de2e43a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.052659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-47640f24-0daf-4a51-a22e-097de2e43a6f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:01.053856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-47640f24-0daf-4a51-a22e-097de2e43a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:01.084840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7f863-dd5c-4c11-9217-26d77d0c6e8c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.122000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.123825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.124222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.124557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.125334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 400/801] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:01.126673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476977a6-418d-4b25-9f63-75174283f8d2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.126965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476977a6-418d-4b25-9f63-75174283f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.127245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476977a6-418d-4b25-9f63-75174283f8d2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.128752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 401/802] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:01.129288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476977a6-418d-4b25-9f63-75174283f8d2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.142499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.144039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-cdf7e7ea-a55f-46d8-a4da-689b8db2f72e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:01.158467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:01.159463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:01.160392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 402/803] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:01.160909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.161941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.161941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.180752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:01.182237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:01.224070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:01.263241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476977a6-418d-4b25-9f63-75174283f8d2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.275569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 403/804] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:01.275569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.289504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:01.315193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.315901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.316189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.316471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.317195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:01.317506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.319453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.336653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 402/805] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:01.336653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.387602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac094acf-9c3e-4d1b-8872-9e53d454a5cd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.423878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.424670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.424954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.425251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.435412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:01.442936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:01.443958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:01.444829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 404/806] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:01.445592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.452805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa355d-4425-46bd-8d9f-874f459644b6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.478546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:01.489143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.489805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.490929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.490929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.491097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:01.491423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.492845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.505336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 403/807] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:01.506354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.539594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:01.562892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bbc24-dc36-41f1-bc65-4945b2f84723 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.605995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.606789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.607836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.608347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.609062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.610419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.612425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3526f-e5b4-4d6b-9a07-06f79d4ae98b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:01.622733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:01.623944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:48:01.624979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:01.629693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 405/808] 10.222.0.58 () {56 vars in 1129 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:01.630380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.655239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.656127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.656413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.656695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.663861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.665356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:01.679860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:01.680761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:01.681638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 404/809] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:01.682120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.739750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:01.746608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc545-2773-4cf1-9fb9-8750551bddff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.780227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:01.783789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.784454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.784738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.785009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.785746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:01.786242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.787421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.788751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0de9b-b6ba-49a2-b788-7dba656e3714 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.801205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 406/810] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:01.801205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.830335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.831258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.831571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.831846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/df08ee888dfd4aeb8025b594fcb2702e/roles/777a339245484c7fbe89538db336a995` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.847728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=df08ee888dfd4aeb8025b594fcb2702e, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.849271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.908518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326900f0-3945-4a8c-a356-18720bde95ef None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:01.930069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 405/811] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:48:01 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/df08ee888dfd4aeb8025b594fcb2702e/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:48:01.931259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:01.955999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:01.956885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:01.957298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:01.957601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:01.958481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:01.958805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:01.961096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:01.972565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 407/812] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:01.972565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.029669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:02.081231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b331-6ce5-4e14-ad28-7a108e0de406 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.099676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552302-509d-46b1-a729-841bdcca9494 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.121174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.122244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.123402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.123909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.124973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:02.125636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.127400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.136834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.138008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.138008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.138284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.142983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 406/813] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:02.144128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.147107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:02.150441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.151925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:02.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:02.172108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:02.173265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:02.173265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:02.173265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:02.173265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:02.173265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 408/814] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:02.173265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.254957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:02.274222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adce9b-3820-4d10-9ab6-d85d34523a60 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.287436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8366a0-d004-46d3-8156-cb54fc89af54 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.316079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.317753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.318551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.319057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.320655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:02.321304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.322957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.325430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.327569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.328173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.328763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/8bd57a7e4c074ca2890ec6bc081809cf/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.331292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 407/815] 10.222.0.58 () {56 vars in 1124 bytes} [Thu Jun 16 17:48:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:02.332428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.336678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=8bd57a7e4c074ca2890ec6bc081809cf, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.338219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.407568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:02.409643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 409/816] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:48:02 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/8bd57a7e4c074ca2890ec6bc081809cf/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:48:02.410596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.459084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4739d-9fe6-49ff-b021-108cb3a014ad None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.510025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.510683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.510972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.511248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.512033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:02.512342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.514130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.529989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 408/817] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:02.537367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.676241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f1224-2196-4370-b3d7-627895272ccd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.699501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bde1ff-dd79-4e7e-977f-75c52d1485cc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.713929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:02.721207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064dfb8-8df3-4fad-8a7a-adff1ade3fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.721883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064dfb8-8df3-4fad-8a7a-adff1ade3fd3 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.722167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064dfb8-8df3-4fad-8a7a-adff1ade3fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.722447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064dfb8-8df3-4fad-8a7a-adff1ade3fd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.723193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064dfb8-8df3-4fad-8a7a-adff1ade3fd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.724597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064dfb8-8df3-4fad-8a7a-adff1ade3fd3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.734940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:02.745692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.749353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.749353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.749353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.756483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.758027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f22bcdff-a488-4d50-886d-030edea0567b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:02.772779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:02.773806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:02.775383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 409/818] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:48:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:02.776152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabcc1d-0eed-4063-aecd-357296beb5a1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.776152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabcc1d-0eed-4063-aecd-357296beb5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.776152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabcc1d-0eed-4063-aecd-357296beb5a1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.778141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 410/819] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:02.779840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabcc1d-0eed-4063-aecd-357296beb5a1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:02.850246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 410/820] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:02.860164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.860461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.860750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.896556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:02.897786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:02.906050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabcc1d-0eed-4063-aecd-357296beb5a1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:02.950621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:02.951642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:02.951979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:02.952256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:02.954393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:02.954749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:02.956920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:02.997145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 411/821] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:48:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:03.004469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.047883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 411/822] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:03.071115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.085727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:03.155271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37aed7-526b-49b0-a83c-bc165197fbbc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:03.194225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:03.195182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:03.197006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:03.197587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:03.199681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3972b-847e-454b-b987-13b4d1ca0efc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:03.221865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:03.243346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:03.245428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:03.246182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:03.246959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] PATH_INFO: `/v3/services/network` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:03.247825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:03.249901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:03.268004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:03.269247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:48:03.270174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:03.272867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 412/823] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:03 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:03.279650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.282997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:03.285290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:03.338632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:03.414530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 412/824] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:03.419413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b6e3-904b-4098-9400-987d6ae51548 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:03.420749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.485090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:03.485817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:03.486054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:03.486404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:03.487037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] RBAC: Adding query filter params (name=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:03.487344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:03.488725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:03.501398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 413/825] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:03.507538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.615716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:03.710463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293174d-f9fb-4a8f-ac0e-3d7a806227d6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:03.736538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c615970-eb08-448c-89e1-d66e3502df8a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:03.781959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:03.784029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:03.784681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:03.786141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:03.798836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:03.802297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:03.804260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:03.822097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:03.823222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:03.824871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:03.826801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:03.827843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:03.828376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:03.830324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:03.845779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 414/826] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:03.846399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:03.847281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:03.848246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:03.848246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:03.848246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:03.848246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:03.851253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.855354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 413/827] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:03.859524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:03.977026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8494c0-3fdc-4f9a-b377-6131f95b11dc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.025833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.026510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.026793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.027068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.030770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:04.031090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:04.032498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:04.041622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913a318-74c9-4d49-8bca-ce6ad877fa6d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.058899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:04.066980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 414/828] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:04.073148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:04.104170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05372825-f909-4761-813a-152ef9e50692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.104838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05372825-f909-4761-813a-152ef9e50692 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.105107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05372825-f909-4761-813a-152ef9e50692 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.105382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05372825-f909-4761-813a-152ef9e50692 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.106119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05372825-f909-4761-813a-152ef9e50692 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:04.108594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05372825-f909-4761-813a-152ef9e50692 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:04.151855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:04.194976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 415/829] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:04.219235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde3b6d-776b-4e4c-b4b3-489daca5baf7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.286097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.286913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.287193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.287498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.296261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:04.297970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d8e6210a-573a-4327-9edb-e49ae21e389e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:04.313168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:04.314120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:04.315126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 415/830] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:04.321906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05372825-f909-4761-813a-152ef9e50692 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:04.416574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443289d4-d590-479a-9307-3299e67e68bf None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.416916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443289d4-d590-479a-9307-3299e67e68bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.417101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443289d4-d590-479a-9307-3299e67e68bf None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.418701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 416/831] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:48:04.425164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.425450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.425723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.457883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:04.459268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:04.469144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05372825-f909-4761-813a-152ef9e50692 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.512089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.512089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.512089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.512089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.512474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:04.512474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:04.514573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:04.530940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 416/832] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:04.537232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:04.561638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 417/833] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:04.567987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:04.589100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:04.646080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:04.657335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f631c-c583-4b8f-8d73-6f72dd7f0fec None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.697371 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.698398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.698974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.699480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.701130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=9279195b1ed64901a63a3fce8d0bb610) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:04.702182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:04.704016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:04.710862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 417/834] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:04 2022] GET /identity/v3/role_assignments?user.id=9279195b1ed64901a63a3fce8d0bb610&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:04.719525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d72d9-c40c-4d02-abdd-64b1a0b90e5c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.760003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.760829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.761108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.761383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.774815 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:04.783551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:04.784552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:04.785387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 418/835] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:04.792547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:04.915615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:04.925876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331149d-b6d2-4678-8e67-3354e75b3ece None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:04.961958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:04.962621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:04.962907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:04.963186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:04.964023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:04.964334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:04.965717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:04.976085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 418/836] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:48:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:04.981755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.085368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.089924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f7676-bcdc-4a52-8fc3-139f95ba3712 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.133788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.135441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.135441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.135817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.151289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:05.152806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-23b67e9c-efb4-4056-a616-2c1bb3d8a1a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:05.172455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:05.173690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:05.174584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:05.174584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:05.174584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:05.174584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:05.174584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 419/837] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:05.180921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.210381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c71d0-8398-43e5-976d-06b7031e3a16 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.210694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c71d0-8398-43e5-976d-06b7031e3a16 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.210940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c71d0-8398-43e5-976d-06b7031e3a16 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.212682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 420/838] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:05.217964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.218249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.218525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.236376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:05.237701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:05.291194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d0d1-40f3-47be-9ec7-3f30c0ca3b9d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.327827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.329804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.330326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.330815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.331817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:05.332334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:05.333960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:05.335979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 421/839] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:05.347958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 419/840] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:05.353880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.356377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.374312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.448279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.466101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622253e-6479-453a-b2c2-b2324a87f659 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.474045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83d671-78cd-4579-860b-c2f122b796c9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.505813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda9417-8445-4718-bd0b-e8522e368882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.506895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda9417-8445-4718-bd0b-e8522e368882 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.507309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda9417-8445-4718-bd0b-e8522e368882 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.507777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda9417-8445-4718-bd0b-e8522e368882 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.515586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.516717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.517146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.517562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.519825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda9417-8445-4718-bd0b-e8522e368882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:05.525473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:05.527084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ffda9417-8445-4718-bd0b-e8522e368882 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:05.527839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:05.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 420/841] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:05.535045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3893d-7822-45f0-bd0e-2e9cef1083d1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.535981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3893d-7822-45f0-bd0e-2e9cef1083d1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.535981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3893d-7822-45f0-bd0e-2e9cef1083d1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.535981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 421/842] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:05.536854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3893d-7822-45f0-bd0e-2e9cef1083d1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b44fb5cb-96a1-4f68-9418-b56e6c040cce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:05.553770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:05.554875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:05.556064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:05.558473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 422/843] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:05.560825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.561728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.580082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:05.581299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:05.620022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.654148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 423/844] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:05.655163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3893d-7822-45f0-bd0e-2e9cef1083d1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.655647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.696856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.697532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.697869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.698155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.698901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:05.699208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:05.700638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:05.709714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 422/845] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:05.710511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.740237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.771925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58136e10-34fb-4d68-a84d-d1bc6a581473 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.811010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.811716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.811998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.812279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.813116 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:05.813425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:05.814842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:05.815612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6cada-7ca7-40a1-ad50-e841b22ab882 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.832366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 424/846] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:05.833190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.861384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.862514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.863009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.863503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.864332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:05.865853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:05.867915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:05.884978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:05.885993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:05.886838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 423/847] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:05.887677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:05.947410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efead0a-eb31-4f53-9ad3-9fd5e7f0ca29 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:05.975661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:05.986587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:05.988063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:05.988251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:05.988467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:05.990612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9162ab-3a2c-43a7-85c7-04b991284401 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.001259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.003517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:06.023335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:06.024298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:06.025197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:06.025197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:06.025197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:06.025197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:06.025197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 425/848] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:06.025482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.031789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.032453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.032827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.033105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.102086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.105554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.107580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.134344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071732a7-8c55-46eb-8f2d-7ff42d2493dd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.171005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.171705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.171997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.172278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.173245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:06.173555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.174914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.180278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 426/849] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:06.184249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.209758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 424/850] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:06.210606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.238380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.301555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbab61-0ccb-47e9-ab96-1ee25cba18de None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.319260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.322848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083cb39-5f86-4db3-8375-7aef1823809b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.338047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.338727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.339017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.339299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.340164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:06.340478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.342454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.358579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 427/851] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:06.358579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.364106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.364772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.365055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.365330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.366031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:06.366336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.367719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.372572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 425/852] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:06.373602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.489387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0be6d1-28ff-42c0-bc8d-76fced15dd5e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.492176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.523584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfdc37b-310a-49a9-91c6-64d3db051cd6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.530765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.531643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.531935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.532218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.545334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.546807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:06.567256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:06.568445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:06.569530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:06.569530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:06.569530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:06.569530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:06.569530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 428/853] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:06.571520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.572515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.572805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.573083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.574830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.584497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.586054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:06.604567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:06.605586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:06.606986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 426/854] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:06.612231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.639662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.692718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350817-fc24-44bb-89fa-2d71069a9f98 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.724977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5841-d333-4743-9fa9-70db1e519fc6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.729704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.730486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.730873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.731253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.732209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:06.732616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.734476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.755271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 429/855] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:06.756350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.766860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.767713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.768000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.768327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.769340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:06.769655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.771179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.778007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.794959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 427/856] 10.222.0.58 () {56 vars in 1135 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:06.796126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:06.893403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6bc35-fabb-47ae-a119-d712b4789560 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.905108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:06.927535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa39212-fde3-448a-97fe-ed0e4554ada5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:06.946608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.947747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.948193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.948659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.957104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:06.959047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:06.970122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:06.970770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:06.971056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:06.971328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:06.974781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:06.975889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:06.976902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:06.984329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 430/857] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:06.985437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.046229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.048140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.071055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.095339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37addcb1-06a7-48df-a32d-7b784f307d83 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.133599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.133824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.134620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.134742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.135019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.135843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:07.136170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.137627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2429b4-9d61-451a-91f2-4f159fffe1f9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.150717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 431/858] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:07.152846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645b9f7-0195-438d-8e8f-4a695df1d194 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.153394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645b9f7-0195-438d-8e8f-4a695df1d194 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.154019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645b9f7-0195-438d-8e8f-4a695df1d194 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.158578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 428/859] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:48:07.159573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 432/860] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:07.159749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.160707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645b9f7-0195-438d-8e8f-4a695df1d194 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.286708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa211-c548-4ffd-ac16-57efa74b3158 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.289212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645b9f7-0195-438d-8e8f-4a695df1d194 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.323697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.324282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.324565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.324842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.325432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.325712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=df08ee888dfd4aeb8025b594fcb2702e) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:07.326030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.326395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.326683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.326964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.327395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e506c63-10a5-4afd-acc7-c1d19309ab20 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.330403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.333449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 429/861] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/role_assignments?user.id=df08ee888dfd4aeb8025b594fcb2702e&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:07.334901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.335206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.335493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.345735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.347639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.363240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:07.364676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:07.373139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:07.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:07.375045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:07.375045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:07.375045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:07.375045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:07.377481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 433/862] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:07.383462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.415482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.451979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 430/863] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:07.474851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.494239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbe8f4-1217-4144-84c5-1c14263fa017 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.538773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.539490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.539776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.540051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.540776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:07.541085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.542430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.556585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 434/864] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:07.562802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.619668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.622939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c6a7-c0fc-4059-94f3-81e5227cf980 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.666215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.666946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.667235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.667552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.668237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.669612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:07.684031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:07.685032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:07.685928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:07.686733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77669d-e8a5-4fe0-badc-dddd9b82c4fb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.686817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 431/865] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:07.691400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.725530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.726341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.726632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.726913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.734580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.736056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.737136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:07.751959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:07.762601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 435/866] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:07.764433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.801870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f2911-c16b-466d-b983-006b7dc17f5c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.836750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.837431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.837720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.838003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.838718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:07.839030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.840439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.846816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 432/867] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:07.852480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.870125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0395f-344c-4b15-b9d9-3742085bf0b2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.880469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:07.907247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:07.908040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:07.908390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:07.908732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:07.909582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:07.909956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:07.911513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:07.924793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 436/868] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:07.931936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:07.989262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127a09e-8f91-4040-ae1c-69ae936c2d3d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:07.991703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:08.037753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:08.039039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:08.039730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:08.040012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:08.040728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:08.041036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:08.042399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847c53d-9435-4e9d-bcbd-2e61b66bd645 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:08.048350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 433/869] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:08.051667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac61f14-3d43-4873-a130-9da2af98c0bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:08.094362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:08.095898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:08.096392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:08.096891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:08.098181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=8bd57a7e4c074ca2890ec6bc081809cf) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:08.098749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:08.100474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fdbca-5734-466f-a1ac-51c8592f37e3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:08.108365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 437/870] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:07 2022] GET /identity/v3/role_assignments?user.id=8bd57a7e4c074ca2890ec6bc081809cf&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:08.711134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec369395-a934-4da1-9dca-002c94f4fa79 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:08.711677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec369395-a934-4da1-9dca-002c94f4fa79 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:08.711757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec369395-a934-4da1-9dca-002c94f4fa79 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:08.713387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 434/871] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:08.719489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1aa13-197a-4a95-9681-ba7750c6ed20 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:08.719785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1aa13-197a-4a95-9681-ba7750c6ed20 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:08.720069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1aa13-197a-4a95-9681-ba7750c6ed20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:08.738936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2ef1aa13-197a-4a95-9681-ba7750c6ed20 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:08.740185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2ef1aa13-197a-4a95-9681-ba7750c6ed20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:08.768147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:08.815879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 438/872] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:08.838220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec369395-a934-4da1-9dca-002c94f4fa79 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:08.950830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec369395-a934-4da1-9dca-002c94f4fa79 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:08.998686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da7585-7f06-451d-92c1-8eaa1da33d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:08.999290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7585-7f06-451d-92c1-8eaa1da33d15 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:08.999608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7585-7f06-451d-92c1-8eaa1da33d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:08.999891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7585-7f06-451d-92c1-8eaa1da33d15 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:09.000650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da7585-7f06-451d-92c1-8eaa1da33d15 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:09.002046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da7585-7f06-451d-92c1-8eaa1da33d15 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:09.074345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 435/873] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:08 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:09.714091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ed41-4610-446c-aa75-52efb37f5424 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:09.714611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ed41-4610-446c-aa75-52efb37f5424 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:09.714725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ed41-4610-446c-aa75-52efb37f5424 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:09.716358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 439/874] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:09.721838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b587-7070-4727-82e2-46e47af3163d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:09.722663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b587-7070-4727-82e2-46e47af3163d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:09.723170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050b587-7070-4727-82e2-46e47af3163d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:09.740787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c050b587-7070-4727-82e2-46e47af3163d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:09.742333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c050b587-7070-4727-82e2-46e47af3163d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:09.767248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:09.821973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 436/875] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:09.847079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785ed41-4610-446c-aa75-52efb37f5424 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:09.957807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785ed41-4610-446c-aa75-52efb37f5424 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:09.994302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:09.995040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:09.995327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:09.995643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.006818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:10.015724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:10.016708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:10.017972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 440/876] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jun 16 17:48:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:10.024904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b587-7070-4727-82e2-46e47af3163d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.127750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050b587-7070-4727-82e2-46e47af3163d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:10.168211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:10.168928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.169215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.169497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.170250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:10.170562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:10.172025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322b82f-844a-4cd4-8d97-141eac387c1d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:10.181199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 437/877] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:48:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:10.188491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.217297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:10.306388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb49242-135f-4475-8f88-346a1872b72f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:10.316054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6626d-fad2-43c5-bb72-1809546df4e0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.316362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6626d-fad2-43c5-bb72-1809546df4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.316645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6626d-fad2-43c5-bb72-1809546df4e0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.318271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 438/878] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:10.323322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.323638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.323920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.341509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:10.342573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:10.348391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:10.349476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.349958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.350462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.352958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:10.363549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:10.365599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:10.384120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:10.385096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:10.385992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:10.385992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:10.385992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:10.385992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:10.389466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 441/879] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:10.399396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.427484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 439/880] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:10.449782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.497331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:10.526594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e75a-5801-429e-a92e-b0b80951b5f6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:10.563167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:10.563774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd6c16-fb63-4205-9390-53090d54741f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:10.563883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.564142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.564423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.565145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:10.565453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:10.566846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e835a5-48b3-44a7-9bb8-18898a8f67e7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:10.579877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 442/881] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:10.581232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9808-f781-4864-8b7f-d8ac39c28f02 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.581530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9808-f781-4864-8b7f-d8ac39c28f02 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.581811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9808-f781-4864-8b7f-d8ac39c28f02 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.583198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 443/882] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:10.586741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9808-f781-4864-8b7f-d8ac39c28f02 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.607374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667b441-66d5-48aa-ad97-c9630fcd3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:10.608134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667b441-66d5-48aa-ad97-c9630fcd3845 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.608419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667b441-66d5-48aa-ad97-c9630fcd3845 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.608702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667b441-66d5-48aa-ad97-c9630fcd3845 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.609421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667b441-66d5-48aa-ad97-c9630fcd3845 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:10.610775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e667b441-66d5-48aa-ad97-c9630fcd3845 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:10.693018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9808-f781-4864-8b7f-d8ac39c28f02 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:10.707411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 440/883] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:10 2022] POST /identity/v3/services => generated 269 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:48:10.707411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.707411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.707411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.714127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:10.728728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:10.730246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:10.748048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:10.754955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:10.756186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.756728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.757228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.767048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:10.769115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:10.787090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:10.788125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:10.789069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:10.791787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 444/884] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:48:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:10.801492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.836625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 441/885] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:10.859920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:10.931820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de12bc-c816-4895-aa39-7241ba356c04 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:10.977327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:10.978440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:10.978792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:10.979397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:10.980309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:10.980819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:10.983278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:10.997544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd965b11-7bc7-4900-826f-03d20736570c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.005126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 445/886] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jun 16 17:48:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:11.010691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.034193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.035978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.043505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.043505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.043505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.043505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.045771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:11.059259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:11.060968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:11.061822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:11.065139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 442/887] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:11.070598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.083781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.130842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4cbd1-2a5c-4716-a288-017a2d268c78 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.167644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c38d67-1567-4c52-8b82-7c3b65b92f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.168364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c38d67-1567-4c52-8b82-7c3b65b92f5a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.168681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c38d67-1567-4c52-8b82-7c3b65b92f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.169110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c38d67-1567-4c52-8b82-7c3b65b92f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.191869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ec878-a62b-41bb-a3e7-d3a65cdc0715 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.230868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.232348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.232975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.233465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.234507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:11.235012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.236801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.243273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 443/888] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:11.248538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c38d67-1567-4c52-8b82-7c3b65b92f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.249700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.249991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c38d67-1567-4c52-8b82-7c3b65b92f5a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.316305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.341656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.382514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b9fed-5bc0-4fbd-a978-a1aefc48b203 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.387725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 446/889] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:11.387725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f4513-f97b-4653-abb1-ecb5108c13c8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.387725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f4513-f97b-4653-abb1-ecb5108c13c8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.387725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f4513-f97b-4653-abb1-ecb5108c13c8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.387725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 447/890] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:11.391139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f4513-f97b-4653-abb1-ecb5108c13c8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.423581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.424361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.424643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.424921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.425625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:11.425936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.427294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246a9a6-801f-47d6-8715-2c39db7098b4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.433928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 444/891] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:11.436297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.436653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.437001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.455387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:11.456937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:11.529489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f4513-f97b-4653-abb1-ecb5108c13c8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.540282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 445/892] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:11.569557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.570659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.570841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.571121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.571432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.585406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.587107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.600015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:11.613338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:11.620682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:11.621654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:11.621654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:11.621654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:11.621654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:11.621654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 448/893] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:11.626253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.645695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.706333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf552b7-0655-4f88-80fe-43030cdfdeed None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.751389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a74b8-fef4-4310-a181-f96217e126b2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.755082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.755923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.756288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.756566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.757280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.758709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:11.775018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:11.776116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:11.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:11.779053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 446/894] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:11.783894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.799531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.800719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.801250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.801761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.802862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:11.803246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.804787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.818065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 449/895] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:11.823642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.886756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.921128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5faff1-02f2-4d13-b4c6-2dc187bd0a84 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.940362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a721d-4bba-4e0d-abed-055ed3242135 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:11.957319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:11.961457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.962126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.962407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.962681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:11.963409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:11.963725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:11.965079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:11.971072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 447/896] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:11.976969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:11.990694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:11.991875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:11.992457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:11.992898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:12.001152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:12.002862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:12.019771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:12.020711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:12.021579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:12.024026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 450/897] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:12.030232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:12.096012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d614804-1238-4b2e-8676-212787713c13 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:12.132911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:12.133977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:12.134525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:12.135063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:12.136010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:12.136550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:12.138692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:12.149374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 448/898] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:12.150582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:12.197278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcb2c8-db8e-4aae-9a66-cdf335af0109 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:12.223258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:12.225394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:12.239168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:12.239877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:12.240165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:12.240436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:12.241384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:12.241526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:12.243193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:12.267991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 451/899] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:12.270351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:12.299675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288567bc-072c-40f8-a5a5-4aef9f9bf743 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:12.359726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:12.360748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:12.367640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:12.368180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:12.369244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] RBAC: Adding query filter params (interface=public, service_id=43d322f1849b4a96af50370ae597558a, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:12.369771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:12.371434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:12.382960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a975a-bef7-4f1b-bb0d-40da8e44c5c1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:12.385034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 449/900] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:48:12 2022] GET /identity/v3/endpoints?service_id=43d322f1849b4a96af50370ae597558a&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:12.386146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:12.419234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:12.420400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:12.420740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:12.421093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:12.421993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] RBAC: Adding query filter params (role.id=788818f843d94f6e93bc573dc74f60c8, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=df08ee888dfd4aeb8025b594fcb2702e) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:12.422778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:12.425113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d0ca41-a60a-425b-8ad2-6a165d962595 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:12.435826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 452/901] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:12 2022] GET /identity/v3/role_assignments?user.id=df08ee888dfd4aeb8025b594fcb2702e&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=788818f843d94f6e93bc573dc74f60c8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:12.512229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b23c2-ba49-4670-ad66-f8812774a855 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:12.527671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:12.551348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e274d-90c9-4499-ad47-46d5252bc64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:12.552575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e274d-90c9-4499-ad47-46d5252bc64e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:12.553051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e274d-90c9-4499-ad47-46d5252bc64e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:12.553549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e274d-90c9-4499-ad47-46d5252bc64e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:12.554486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e274d-90c9-4499-ad47-46d5252bc64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:12.556157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e274d-90c9-4499-ad47-46d5252bc64e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:12.565143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 450/902] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:12 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:48:13.008511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a3b40-ead9-4421-86ae-e994aa9e3d4a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.009067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a3b40-ead9-4421-86ae-e994aa9e3d4a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.009067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a3b40-ead9-4421-86ae-e994aa9e3d4a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.010675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 453/903] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:13.016592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.017039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.017139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.035476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:13.036679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:13.113236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 451/904] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:13.136756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a3b40-ead9-4421-86ae-e994aa9e3d4a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:13.188914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:13.249227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a3b40-ead9-4421-86ae-e994aa9e3d4a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:13.285728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:13.286461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.286748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.287023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.287707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:13.289070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:13.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:13.303891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:13.305142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 454/905] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Jun 16 17:48:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:13.310809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:13.421318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885dcad-74f0-42a1-ab96-4a0fd885dccb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:13.429458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:13.461398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:13.462096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.462376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.462653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.463390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:13.463707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:13.465094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:13.473339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 452/906] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:13.478718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:13.583683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8c62d-e08e-4667-a7c2-a09d2e913639 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:13.593228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:13.618937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:13.619641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.619931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.620212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.620924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:13.621236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:13.622848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731dd233-00a2-4cdb-a1a7-4a01783a3161 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:13.628184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 455/907] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:13.633964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:13.724175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2cf28-e270-4da3-a652-dd24bceb4acf None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.724499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2cf28-e270-4da3-a652-dd24bceb4acf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.724744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2cf28-e270-4da3-a652-dd24bceb4acf None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.726410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 456/908] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:13.731613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.731904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.732259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.749299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:13.750495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:13.750946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700f3e2-b4d6-4be7-b329-8190aa1200d2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:13.799649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:13.801120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:13.801850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:13.802320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:13.803582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] RBAC: Adding query filter params (interface=public, service_id=5e55025016ec405f977f9aed1b5dfd0a, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:13.804653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:13.806761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:13.818943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 453/909] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:48:13 2022] GET /identity/v3/endpoints?service_id=5e55025016ec405f977f9aed1b5dfd0a&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:13.820096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:13.837326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 457/910] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:13.860180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:13.865549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:13.954350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49abec1-5b71-4dc7-897b-ed45247ed603 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:13.978661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1471509-6115-4fe4-a9eb-d5f374f1b5bc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:13.980607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:13.999743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d411a3-91b4-480c-b6d2-b6a8a389f477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:14.001293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d411a3-91b4-480c-b6d2-b6a8a389f477 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:14.001808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d411a3-91b4-480c-b6d2-b6a8a389f477 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:14.002309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d411a3-91b4-480c-b6d2-b6a8a389f477 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:14.003193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d411a3-91b4-480c-b6d2-b6a8a389f477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:14.006552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d411a3-91b4-480c-b6d2-b6a8a389f477 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:14.023622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117abf6-fa5d-46d8-8382-b14df78452b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:14.023622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117abf6-fa5d-46d8-8382-b14df78452b0 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:14.023622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117abf6-fa5d-46d8-8382-b14df78452b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:14.023622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117abf6-fa5d-46d8-8382-b14df78452b0 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:14.027506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117abf6-fa5d-46d8-8382-b14df78452b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:14.027506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117abf6-fa5d-46d8-8382-b14df78452b0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:14.027703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 454/911] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:13 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:48:14.121031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 458/912] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:13 2022] POST /identity/v3/services => generated 241 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:14.892545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497dcc8-df22-4b38-98f9-15418e801462 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:14.892927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497dcc8-df22-4b38-98f9-15418e801462 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:14.893138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497dcc8-df22-4b38-98f9-15418e801462 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:14.894706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 455/913] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:14.900757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e3eb7-5ce3-46ba-abbd-b849aab713a1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:14.901042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e3eb7-5ce3-46ba-abbd-b849aab713a1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:14.901320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e3eb7-5ce3-46ba-abbd-b849aab713a1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:14.902316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:14.902822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 456/914] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:14.903119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:14.903734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:14.911670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:14.912226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:14.912723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:14.921119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:14.922598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:14.930353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:14.931934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:15.019410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 457/915] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:15.028564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 459/916] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:15.041769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.054462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.071865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.169889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e4aca-fcde-4597-83b6-efc77212ed91 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.171878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.189933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55541b-724c-4508-aa73-1ffa33dadc79 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.210281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.211170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.211486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.211767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.212751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.214201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:15.228368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:15.229397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:48:15.230244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:15.232661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.233716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.234223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.234708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.235346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 458/917] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:15.236416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.246870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:15.256406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:15.257355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:15.258226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:15.270855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 460/918] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:15.275842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.343400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.361909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b447d-be1e-408c-b622-3e2e315cc434 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.392910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6769-f96b-4b51-9b97-88ad4baac554 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.410385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.411326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.411640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.411925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.412635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:15.412945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.414973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.422644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 459/919] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:15.427546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.436912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.437811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.438241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.438662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.439672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:15.440126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.441638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.455715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 461/920] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:15.457291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.471265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.534387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2531c-380e-4b07-8094-690e520fd21c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.569262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5369b-76b0-49de-b57e-f7482a3cd803 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.569534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.570215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.570500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.570783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.571530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:15.571907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.573507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.584140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 460/921] 10.222.0.58 () {56 vars in 1114 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:15.585001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.603673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.610470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.611547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.612051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.612492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.624874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.626546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:15.646693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:15.647876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:15.648996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:15.648996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:15.648996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:15.648996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:15.655577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 462/922] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:15.656936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.690721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a0e8f-ec58-4336-9089-b0916327a836 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.725857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.726790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.727495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.727787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.728067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.728813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.730181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.770646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f934-4831-4b69-8e85-50b2a4bf3077 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:15.805777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b548-24f9-4407-9633-852191069830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:15.806450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b548-24f9-4407-9633-852191069830 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:15.806737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b548-24f9-4407-9633-852191069830 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:15.807014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986b548-24f9-4407-9633-852191069830 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:15.807767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b548-24f9-4407-9633-852191069830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:15.808183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b548-24f9-4407-9633-852191069830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:15.810138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986b548-24f9-4407-9633-852191069830 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:15.831510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 461/923] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:15.839482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 463/924] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:15.844227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:15.926518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:15.968630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a81497-3f7a-444d-92e5-11ea0ac00aae None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.008166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.009423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.009961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.010451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.020251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.022073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f2a9900e-6a79-4571-94ec-488dca0bb7ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:16.038344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:16.043818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:16.044794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 462/925] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:48:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:16.046172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b548-24f9-4407-9633-852191069830 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.162896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986b548-24f9-4407-9633-852191069830 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.189590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:16.203152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.204106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.204689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.206349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4603c-b5cd-4336-9251-74e3ba9e4960 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.206632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4603c-b5cd-4336-9251-74e3ba9e4960 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.206904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4603c-b5cd-4336-9251-74e3ba9e4960 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.207395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.208497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:16.209034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.210671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.212027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 463/926] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:48:16.213703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.214008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.214840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.233490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:16.235030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:16.243440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 464/927] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:16.248273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.318178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 464/928] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:16.344285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.383857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4227cd8-5693-4814-90e6-de51d4654405 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.414009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:16.423648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.424502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.424734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.425012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.435603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.437049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-45d29991-6a11-45ea-80be-e02534b514fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:16.455653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:16.456585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:16.457462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:16.457462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:16.457462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:16.457462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:16.457704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5311ba-bfac-4a4b-8c8a-744f2db9a9c2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.461573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 465/929] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:16.463007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74815227-a6ee-4a68-9e53-8f8531b94015 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.463299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74815227-a6ee-4a68-9e53-8f8531b94015 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.463606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74815227-a6ee-4a68-9e53-8f8531b94015 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.465606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 466/930] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:16.466138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74815227-a6ee-4a68-9e53-8f8531b94015 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.499738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.500460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.500745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.501020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.501671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.502030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:16.503544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-beb2fd3d-ee39-4bbe-b443-5502568ae55d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:16.516046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:16.517731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:48:16.518887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:16.522303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 465/931] 10.222.0.58 () {56 vars in 1132 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:16.523625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.523917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.524198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.541048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:16.542236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:16.585248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74815227-a6ee-4a68-9e53-8f8531b94015 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.612640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 466/932] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:16.613491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.633701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.634389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.634682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.634960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.636017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:16.636343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.638386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.651085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 467/933] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:16.651787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.683939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:16.733540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e93cf-671d-48ef-97be-2f27efbf1fe3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.754965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:16.764788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589724bb-2138-417e-99e7-e454155e3774 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.772027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.774512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.774512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.774512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.774512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:16.774512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.775650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.785690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 467/934] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:16.785690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.810770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.811734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.812128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.812504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.813416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.815065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:16.828752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:16.830374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:16.831488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 468/935] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:16.832466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.891953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122bb1e-2f75-4c0b-b36e-37c3d2997f0b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.908835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:16.927791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.928584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.928877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.929194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.936331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.937758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.938442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1dfb4-244c-4265-887a-44f223e9d967 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:16.954056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:16.955144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 468/936] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:16.956088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:16.976990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:16.977661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:16.977947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:16.978283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:16.978932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:16.979238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:16.980642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:16.990222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 469/937] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:16.990222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:17.013895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:17.081388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeef322-2afc-41c0-bf53-a94a60ee4808 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.100445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c69bf-a508-482e-9aad-f454cd2738e5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.119261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.119971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.120261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.120542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.121250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:17.122489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.123889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.128785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 469/938] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:17.129530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:17.141999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.142677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.142966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.143249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.144125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:17.144442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.145863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.163442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 470/939] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:17.163442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:17.178492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:17.246765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:17.257573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fc3b-977b-48b8-a7cb-b3b58340c6ac None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.268085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92bf81-0c80-490a-a0b6-9462d3aee889 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.297550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.298717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.299174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.299486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.300292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.302171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.307060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.307861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.308152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.308326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.309537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:17.309847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.311209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.316304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 471/940] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:17.317068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:17.384639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 470/941] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:17.385469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:17.451938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f179038-9389-450b-87dc-7e18446c8165 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.515217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:17.524315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ec00-3a7c-466d-8148-cdeafa1d2c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.525418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862406eb-9672-462d-a5a4-20d6d69cf5e3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.526234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ec00-3a7c-466d-8148-cdeafa1d2c6f None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.528599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ec00-3a7c-466d-8148-cdeafa1d2c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.529009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ec00-3a7c-466d-8148-cdeafa1d2c6f None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/df08ee888dfd4aeb8025b594fcb2702e/roles/788818f843d94f6e93bc573dc74f60c8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.537596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457ec00-3a7c-466d-8148-cdeafa1d2c6f None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=df08ee888dfd4aeb8025b594fcb2702e, role_id=788818f843d94f6e93bc573dc74f60c8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.540358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457ec00-3a7c-466d-8148-cdeafa1d2c6f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.569212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.569895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.570834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.570834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.571254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] RBAC: Adding query filter params (interface=public, service_id=00963c406e8c4e92ab199f9470f5b1d4, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:17.571669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.573069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.579217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 471/942] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:48:17 2022] GET /identity/v3/endpoints?service_id=00963c406e8c4e92ab199f9470f5b1d4&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:17.585421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:17.624033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:17.629057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 472/943] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:48:17 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/df08ee888dfd4aeb8025b594fcb2702e/roles/788818f843d94f6e93bc573dc74f60c8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:48:17.706440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4f813-c7e1-48ef-b19a-3d91c32044e3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:17.751852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e780c-94f5-48d8-8020-1ee28eec17c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:17.752538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e780c-94f5-48d8-8020-1ee28eec17c2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:17.752826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e780c-94f5-48d8-8020-1ee28eec17c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:17.753106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e780c-94f5-48d8-8020-1ee28eec17c2 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:17.753797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e780c-94f5-48d8-8020-1ee28eec17c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:17.755201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e780c-94f5-48d8-8020-1ee28eec17c2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:17.764145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 472/944] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:17 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:48:18.154616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f8e08-3471-4dd6-b9fd-ce693d8fd183 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:18.155133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f8e08-3471-4dd6-b9fd-ce693d8fd183 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:18.155209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f8e08-3471-4dd6-b9fd-ce693d8fd183 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:18.156809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 473/945] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:18.163876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:18.164315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:18.164646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:18.182148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:18.183353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:18.203116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:18.261079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 473/946] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:18.284963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f8e08-3471-4dd6-b9fd-ce693d8fd183 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:18.397209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f8e08-3471-4dd6-b9fd-ce693d8fd183 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:18.433917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:18.434851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:18.435291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:18.435739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:18.436600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:18.438187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:18.439759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING keystone.server.flask.common [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 17:48:18.523049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.sql.core [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:48:18.523049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:48:18.523049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [parameters: {'id': '8d70c8acdfd24fab887dc751f6cbab5a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:48:18.523049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d70c8acdfd24fab887dc751f6cbab5a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d70c8acdfd24fab887dc751f6cbab5a. Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:48:18.531824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 17:48:18.532626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application yield resource Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise self.value Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application session.commit() Jun 16 17:48:18.533579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:48:18.534460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:48:18.535384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:48:18.537244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [parameters: {'id': '8d70c8acdfd24fab887dc751f6cbab5a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:18.539357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:18.541302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d70c8acdfd24fab887dc751f6cbab5a. Jun 16 17:48:18.544081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:18.547354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 474/947] 10.222.0.58 () {60 vars in 1140 bytes} [Thu Jun 16 17:48:18 2022] POST /identity/v3/projects => generated 262 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:18.552917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:18.659005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e117f9-cf5e-4ee0-b6b2-1706ff5df983 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:18.698915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:18.701148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:18.701661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:18.702154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:18.709536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:18.711171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d367cca8-c47a-48bc-923a-517c1ae0375c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:18.725735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:18.726636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:18.727549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:18.729974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 474/948] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:18.735800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:18.796475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:18.846774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eda34-995b-497a-9db1-514b1a3ba82b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:18.883544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:18.884561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:18.885066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:18.885535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:18.886497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:18.887659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:18.889488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea5da4-8c79-454a-aaf2-ebed8d2f5013 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:18.910995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 475/949] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:19.663787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304333ed-17d7-4b93-82ad-85401f9218e4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:19.664279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304333ed-17d7-4b93-82ad-85401f9218e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:19.664279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304333ed-17d7-4b93-82ad-85401f9218e4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:19.665896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 475/950] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:19.672418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b28b7-310d-4606-aa89-a490bfabb496 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:19.672816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b28b7-310d-4606-aa89-a490bfabb496 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:19.673029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b28b7-310d-4606-aa89-a490bfabb496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:19.690921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8e4b28b7-310d-4606-aa89-a490bfabb496 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:19.692487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8e4b28b7-310d-4606-aa89-a490bfabb496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:19.767531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 476/951] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:19.786214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304333ed-17d7-4b93-82ad-85401f9218e4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:19.804834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:19.907107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304333ed-17d7-4b93-82ad-85401f9218e4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:19.944718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:19.945968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:19.946460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:19.946939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:19.947886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:19.950688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:19.958017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2531c-fe2a-476d-9ec4-5ab0721a4be4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:19.958347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2531c-fe2a-476d-9ec4-5ab0721a4be4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:19.958589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2531c-fe2a-476d-9ec4-5ab0721a4be4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:19.961629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 477/952] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:19.965579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:19.967359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:19.968318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:19.969096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bc527-3acc-47a3-8024-76b09df36000 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:19.969096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bc527-3acc-47a3-8024-76b09df36000 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:19.969096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bc527-3acc-47a3-8024-76b09df36000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:19.972234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 476/953] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:19.978331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:19.990234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-249bc527-3acc-47a3-8024-76b09df36000 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:19.994401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-249bc527-3acc-47a3-8024-76b09df36000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:20.042850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.069214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 478/954] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:20.070578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a098af7-6678-422f-9401-42495d642229 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.070870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a098af7-6678-422f-9401-42495d642229 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.071150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a098af7-6678-422f-9401-42495d642229 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.072632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 479/955] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:20.078628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.079412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.079412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.084388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.103353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:20.104643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:20.105940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d75efd7-482a-442c-9519-a5f6672251e4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.147019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.147750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.148036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.148312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.149095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:20.149323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.150694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.155888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 477/956] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:20.156672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.180448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 480/957] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:20.181211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.279684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f8c97-576f-4983-926b-d347042ee034 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.303332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907ff97-cd02-40bf-8ea8-332774a75ca2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.312664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.313429 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.314242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.314547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.314819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.325741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:20.334053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:20.335167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 478/958] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:20.341266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.344406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.345088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.345568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.346183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.346448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.347397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:20.348099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.349897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.354856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 481/959] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:20.356381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.471855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f148a-9f8e-44ee-aa2e-316f7a951ab2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.493887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f46ad-04ed-4067-ae05-5ceca8a4a9cd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.521604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.523189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.523689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.524693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.526216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.528817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.545755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:20.546394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:20.547681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:20.548533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:20.549274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.549274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.549274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.549274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:20.549516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 479/960] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:20.549585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.550192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.553692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.565977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 482/961] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:20.572361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.619746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.647563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.698868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20562-40a2-4ccf-8cb6-06cbd83e9ade None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.700892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467cc0a-eda6-4bfa-999c-85322d196fcb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.739062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.739917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.740203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.740479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.751431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.752673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.752784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.753328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.753806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.754191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.755108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:20.755640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.757947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.765432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 480/962] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:20.771301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:20.772274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:20.773135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:20.774024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:20.774024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:20.774024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:20.774024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:20.779830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 483/963] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:20.780082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.886371 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.890854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88260d61-9572-478f-8a98-3e6523387ba5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.908362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92164402-3dda-40eb-ae3a-e9d17f87a655 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:20.922529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:20.934641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.935490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.938964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.939487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.940644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:20.941138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.943580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.952252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 481/964] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:20.957962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:20.959805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:20.960560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:20.960924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:20.961283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:20.962186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:20.962574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:20.964198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:20.986240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 484/965] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:20.987303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.088461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9924ee5-656e-4a9e-b30c-8192c7cef0ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.112220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455bfbb-fad4-4e0a-8663-fddf2959bcba None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.129677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd89c81-2110-496d-8cd8-3ee2dc0c11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.130456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89c81-2110-496d-8cd8-3ee2dc0c11a9 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.130819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89c81-2110-496d-8cd8-3ee2dc0c11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.131164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89c81-2110-496d-8cd8-3ee2dc0c11a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.132041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd89c81-2110-496d-8cd8-3ee2dc0c11a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.133540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd89c81-2110-496d-8cd8-3ee2dc0c11a9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.154782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.155936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.156384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.164124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.164409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.165857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7208836c-fb47-4afd-822e-395e23173ecc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:21.182789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:21.184543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 485/966] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:48:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d520e1-d1fa-4ff2-8919-3f0426fa9bad None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.190501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d520e1-d1fa-4ff2-8919-3f0426fa9bad None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.191301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d520e1-d1fa-4ff2-8919-3f0426fa9bad None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.191301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 486/967] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:21.192474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d520e1-d1fa-4ff2-8919-3f0426fa9bad None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.197202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:21.225143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 482/968] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:48:21.233381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.233749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.234151 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.262022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:21.263810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:21.319942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:21.324587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d520e1-d1fa-4ff2-8919-3f0426fa9bad None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.356047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 483/969] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:21.388596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.388596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.388596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.389155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.390444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:21.391011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.393942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.400821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.421513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 487/970] 10.222.0.58 () {56 vars in 1149 bytes} [Thu Jun 16 17:48:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:21.427103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.498791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:21.521406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263741e0-bc46-4024-b626-ed35587dd51d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.553020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9380b6f-3299-4b35-a1fc-abd5ec8112cc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.565856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.566679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.567141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.567547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.568339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.569976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.571495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING keystone.server.flask.common [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 17:48:21.592673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.593344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.593635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.593933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.615876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.sql.core [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:48:21.615876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:48:21.615876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [parameters: {'user_id': '6edeb60a5d0d451bb36311ae4030c994', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:48:21.615876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80353) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jun 16 17:48:21.634598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:48:21.635598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application yield resource Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise self.value Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application session.commit() Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:48:21.636593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:48:21.637495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:48:21.638406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:48:21.639254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [parameters: {'user_id': '6edeb60a5d0d451bb36311ae4030c994', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:21.641978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 17:48:21.643950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 484/971] 10.222.0.58 () {60 vars in 1131 bytes} [Thu Jun 16 17:48:21 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:21.646132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.682231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.683847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.690325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:21.749652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:21.750220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6742a-5ee8-4a52-9f1e-abe00d87e800 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.786577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.787545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.787899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.788183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.796477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.797751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 488/972] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:48:21.805083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.805242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:21.814420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:21.815676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:48:21.816847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:21.817907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 485/973] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:48:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:21.823530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:21.930320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192230b6-8804-49cb-a919-98ab4e6d3271 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.939436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad7a9b-6e54-4fc8-b649-a8b2d0e640d1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:21.941251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:21.975894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.976836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.977197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.977547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:21.977681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.978352 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:21.978472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:21.978657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:21.978894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.979078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:21.981151 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:21.990868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:21.999693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:22.014268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:22.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:22.016622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:22.016622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:22.016622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:22.016622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:22.020250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 489/974] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:22.021799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:22.035709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 486/975] 10.222.0.58 () {56 vars in 1101 bytes} [Thu Jun 16 17:48:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:22.036285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:22.138490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc80b9-36f6-49fb-8ef3-9f0b68c08b19 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:22.177728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:22.178470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.178770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.179058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.179839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:22.180157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:22.181622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:22.195590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 490/976] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:22.201971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:22.325368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d100f40-f54d-4314-bd93-ebeeb321b529 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:22.361549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:22.362347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.362632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.362906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.369805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:22.371295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-43439e14-45e3-43eb-afee-635c5e05cf48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:22.384916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:22.385825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:22.386793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 487/977] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:22.392019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:22.464056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:22.503196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf64487-3677-4788-ba9e-b16d8ffb822c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:22.541080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:22.542012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.542324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.542706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.544263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:22.544583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:22.544842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd0ad9-3b7f-4625-abf2-d5a97a2ff1c0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.545135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd0ad9-3b7f-4625-abf2-d5a97a2ff1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.545416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd0ad9-3b7f-4625-abf2-d5a97a2ff1c0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.546041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:22.547439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 488/978] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:48:22.552541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.552857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.553111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.563415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 491/979] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:22.568512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:22.571919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:22.573455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:22.588904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:22.651746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 489/980] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:22.671328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:22.675673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef222011-0067-4faf-9c2c-2e12902105a7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:22.705616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:22.712417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:22.713080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.713368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.713654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.724038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] RBAC: Adding query filter params (role.id=788818f843d94f6e93bc573dc74f60c8, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=df08ee888dfd4aeb8025b594fcb2702e) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:22.724038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:22.724038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64767579-8565-4613-8226-fc27f7db05ca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:22.724038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 492/981] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:22 2022] GET /identity/v3/role_assignments?user.id=df08ee888dfd4aeb8025b594fcb2702e&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=788818f843d94f6e93bc573dc74f60c8 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:22.781180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363a8da-a21b-43c4-9aa5-985f6275eb77 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:22.818308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4428d9a-9863-4367-bffc-bfd1b3e9ba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:22.818992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4428d9a-9863-4367-bffc-bfd1b3e9ba20 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:22.819282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4428d9a-9863-4367-bffc-bfd1b3e9ba20 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:22.819596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4428d9a-9863-4367-bffc-bfd1b3e9ba20 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:22.820332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4428d9a-9863-4367-bffc-bfd1b3e9ba20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:22.821715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4428d9a-9863-4367-bffc-bfd1b3e9ba20 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:22.897538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 490/982] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:22 2022] POST /identity/v3/services => generated 242 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:48:23.591241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de918a28-7e55-4086-957d-5121542b5710 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:23.591786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de918a28-7e55-4086-957d-5121542b5710 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:23.591786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de918a28-7e55-4086-957d-5121542b5710 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:23.593149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 493/983] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:23.598146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:23.598440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:23.598716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:23.616124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:23.617327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:23.622136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:23.693874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 491/984] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:23.716816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de918a28-7e55-4086-957d-5121542b5710 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:23.819215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de918a28-7e55-4086-957d-5121542b5710 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:23.854100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:23.854854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:23.855986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:23.856220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:23.856976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:23.858378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:23.871148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:23.872415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:23.873255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:23.874626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 494/985] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:23.878272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:23.979243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a48f-bf96-49dc-b3bf-1e06161efcf1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:24.014940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:24.015659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:24.015948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:24.016227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:24.016953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:24.017264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:24.018674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:24.021156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:24.030580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 492/986] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:24.031822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:24.093564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:24.135805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608e4ab-87b2-4f00-8c78-4b807c9181af None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:24.171160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:24.172098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:24.172389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:24.172662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:24.173358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:24.173662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:24.175008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:24.185035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 495/987] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:24.186177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:24.295801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f14ed-44c5-4ea3-8470-160e710074f5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:24.332931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:24.333600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:24.333886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:24.334165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:24.334956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] RBAC: Adding query filter params (interface=admin, service_id=00963c406e8c4e92ab199f9470f5b1d4, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:24.335271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:24.336712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57ad0a-5984-4620-864e-b930d04bc2c9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:24.349669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 493/988] 10.222.0.58 () {56 vars in 1249 bytes} [Thu Jun 16 17:48:24 2022] GET /identity/v3/endpoints?service_id=00963c406e8c4e92ab199f9470f5b1d4&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:24.349840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:24.453749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b8f7f-18e8-47e0-a935-079c00966c33 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:24.476833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:24.489309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25574441-fe69-4369-898a-74b9f66843a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:24.490185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25574441-fe69-4369-898a-74b9f66843a7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:24.490651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25574441-fe69-4369-898a-74b9f66843a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:24.491084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25574441-fe69-4369-898a-74b9f66843a7 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:24.491988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25574441-fe69-4369-898a-74b9f66843a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:24.503381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25574441-fe69-4369-898a-74b9f66843a7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:24.503381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 496/989] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:24 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:48:24.975875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d990175-ec90-4788-837d-739ab6430349 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:24.976404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d990175-ec90-4788-837d-739ab6430349 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:24.976404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d990175-ec90-4788-837d-739ab6430349 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:24.977796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 494/990] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:24.983816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87a3e-5275-4fd3-9612-d77d1c28793f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:24.984266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87a3e-5275-4fd3-9612-d77d1c28793f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:24.984468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e87a3e-5275-4fd3-9612-d77d1c28793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.002635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-00e87a3e-5275-4fd3-9612-d77d1c28793f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:25.003873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-00e87a3e-5275-4fd3-9612-d77d1c28793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:25.098227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 497/991] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:25.114072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9c64-3425-41da-898f-67823c34d519 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.114342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9c64-3425-41da-898f-67823c34d519 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.114629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e9c64-3425-41da-898f-67823c34d519 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.116202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 495/992] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:25.121821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e9c64-3425-41da-898f-67823c34d519 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.121990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.122239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.122526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.142935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:25.144170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:25.192834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:25.217846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 498/993] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:25.234586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e9c64-3425-41da-898f-67823c34d519 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.239858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.277058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.278122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.278415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.278694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.279355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.280751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.289084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:25.293103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:25.294137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:25.295639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 496/994] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:25.300254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.357710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0692c-8acc-44cd-a736-5ac38c7d39ee None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.413781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.415055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.415557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.416016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.417097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.419385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:25.440335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:25.441338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:25.442175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:25.442942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e39226-2ea6-4153-8974-61d98742305a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.450864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 499/995] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:25.452025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.489390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:25.502006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.503262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.503768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.504230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.505495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] RBAC: Adding query filter params (name=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:25.506043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.508022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.516555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 497/996] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:25.522176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.585466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c412b-0abd-4ea4-ab4f-099fb112819e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.597806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:25.623804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.624642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.624903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.625217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.625983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:25.626884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.629129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.639551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 500/997] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:25.639705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f63b1-b87e-4b46-99ab-265496e05481 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.640897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.677914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.679535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.679824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.680099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.680805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:25.681110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.682544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.689045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 498/998] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:25.762456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5fcf8-bb08-4883-a17f-b3666a02345d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.798890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.799614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.799903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.800182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.800896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:25.801206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.802583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.813500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 501/999] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:25.815323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:25.877947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:25.922670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deeefa-71df-47fc-acff-cf4e5a022759 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:25.956739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:25.957399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:25.957684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:25.957965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:25.958750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] RBAC: Adding query filter params (interface=public, service_id=6628f33553d94a6bb9fd741ff11f5f02, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:25.959064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:25.960458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319443a-4879-43de-b1e4-0fe7d5e4b228 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:25.966468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 499/1000] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/endpoints?service_id=6628f33553d94a6bb9fd741ff11f5f02&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:25.971913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:26.012883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:26.076384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2fe58-4f0b-4ad5-aef9-c8ce9ef033fa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:26.111146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508933c-6756-430a-81e4-41ad2d15d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:26.111844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508933c-6756-430a-81e4-41ad2d15d441 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:26.112130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508933c-6756-430a-81e4-41ad2d15d441 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:26.112409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508933c-6756-430a-81e4-41ad2d15d441 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:26.113093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508933c-6756-430a-81e4-41ad2d15d441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:26.114454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508933c-6756-430a-81e4-41ad2d15d441 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:26.120967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 502/1001] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:25 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:48:26.689187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6484d-53c9-4315-a98d-0d4adc560e1c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:26.689713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6484d-53c9-4315-a98d-0d4adc560e1c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:26.689713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6484d-53c9-4315-a98d-0d4adc560e1c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:26.691244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 500/1002] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:26.696367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:26.696669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:26.696953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:26.713895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:26.715676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:26.786084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 503/1003] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:26.810209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6484d-53c9-4315-a98d-0d4adc560e1c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:26.914742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6484d-53c9-4315-a98d-0d4adc560e1c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:26.936171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:26.953717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:26.953717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:26.953717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:26.953717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:26.954059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:26.954059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:26.966615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:26.967738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:48:26.968621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:26.971449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 501/1004] 10.222.0.58 () {56 vars in 1126 bytes} [Thu Jun 16 17:48:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:26.974105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:27.005822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:27.075710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e62756-306d-4bbe-a259-2408cb863576 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:27.114839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:27.115541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:27.115831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:27.116109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:27.116868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:27.117174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:27.118531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:27.123768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 504/1005] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:27.129206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:27.238373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b4197-057f-4a41-b0fc-2b40d40a057f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:27.273944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:27.274614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:27.274900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:27.275179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:27.275920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:27.276231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:27.277662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b392feb-efac-4abd-984c-4463bdf1eaf9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:27.283318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 502/1006] 10.222.0.58 () {56 vars in 1122 bytes} [Thu Jun 16 17:48:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:27.288788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:27.396776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:27.403717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445f1b5-b22c-4735-b208-d70d63ef3001 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:27.437723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b23b50-4a2a-472b-9a8e-7556a5acfc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:27.438852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b23b50-4a2a-472b-9a8e-7556a5acfc42 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:27.439406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b23b50-4a2a-472b-9a8e-7556a5acfc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:27.440178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b23b50-4a2a-472b-9a8e-7556a5acfc42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:27.440938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b23b50-4a2a-472b-9a8e-7556a5acfc42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:27.442304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b23b50-4a2a-472b-9a8e-7556a5acfc42 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:27.559789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 505/1007] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:48:27.907906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82438c-66db-470e-9ab8-7d21fc0836d5 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:27.908421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82438c-66db-470e-9ab8-7d21fc0836d5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:27.908790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82438c-66db-470e-9ab8-7d21fc0836d5 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:27.911178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 503/1008] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:27.918761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9963-63f8-475e-be41-9b79ddf12d6a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:27.919048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9963-63f8-475e-be41-9b79ddf12d6a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:27.919328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9963-63f8-475e-be41-9b79ddf12d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:27.939642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e31f9963-63f8-475e-be41-9b79ddf12d6a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:27.940838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e31f9963-63f8-475e-be41-9b79ddf12d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:28.071574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 506/1009] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:28.097328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82438c-66db-470e-9ab8-7d21fc0836d5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:28.162780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:28.222972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82438c-66db-470e-9ab8-7d21fc0836d5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:28.266038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:28.266787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:28.267127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:28.267486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:28.268208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:28.269582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:28.359949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a2aa9-7a52-4c01-a513-543cd657cdc1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:28.360481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a2aa9-7a52-4c01-a513-543cd657cdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:28.360567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a2aa9-7a52-4c01-a513-543cd657cdc1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:28.361934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 507/1010] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:28.367644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:28.368042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:28.368482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:28.376767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 504/1011] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:28 2022] POST /identity/v3/services => generated 242 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:48:28.388938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:28.390224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:28.487406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 508/1012] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:28 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:28.510210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:28.645701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0e96f-8526-4e89-b6b2-53d86deb8a5b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:28.654553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:28.678744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:28.679582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:28.679859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:28.680138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:28.680787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:28.682148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:28.693879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:28.694953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:28.696552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 505/1013] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:28.702909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:28.711695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:28.816879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7001e-ad23-479b-a809-e47351590f2a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:28.853623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:28.854758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:28.855408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:28.856008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:28.857029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:28.857596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:28.859197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:28.870089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 509/1014] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:28.871546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:28.976668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145bb95-9e53-4ca8-b829-282d4ef0d1f6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:29.012635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:29.013304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:29.013595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:29.013863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:29.014567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:29.014899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:29.016328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ad9f-7115-4ce0-b40e-2af61535b976 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:29.021438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 506/1015] 10.222.0.58 () {56 vars in 1112 bytes} [Thu Jun 16 17:48:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:29.027346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:29.113733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:29.150758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0479bf6-0511-4cc7-8bea-8fe8c5b0e315 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:29.186582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a0142-a0ca-4510-ad6f-d3d0d549128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:29.187241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a0142-a0ca-4510-ad6f-d3d0d549128d None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:29.187554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a0142-a0ca-4510-ad6f-d3d0d549128d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:29.187832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a0142-a0ca-4510-ad6f-d3d0d549128d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:29.188569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a0142-a0ca-4510-ad6f-d3d0d549128d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:29.189918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a0142-a0ca-4510-ad6f-d3d0d549128d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:29.273574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 510/1016] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:29.925866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c55d6-f5f7-4573-bfe8-6a1415f4f19b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:29.926245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c55d6-f5f7-4573-bfe8-6a1415f4f19b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:29.926429 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c55d6-f5f7-4573-bfe8-6a1415f4f19b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:29.927981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 507/1017] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:29.933506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:29.933966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:29.934131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:29.951393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:29.952591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:30.027655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 511/1018] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:30.050710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c55d6-f5f7-4573-bfe8-6a1415f4f19b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:30.092126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:30.160536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c55d6-f5f7-4573-bfe8-6a1415f4f19b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:30.196676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ac4af-f468-43ea-ab58-d0abbdcfafa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:30.197383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ac4af-f468-43ea-ab58-d0abbdcfafa7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:30.197710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ac4af-f468-43ea-ab58-d0abbdcfafa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:30.198063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ac4af-f468-43ea-ab58-d0abbdcfafa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:30.199864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ac4af-f468-43ea-ab58-d0abbdcfafa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:30.201289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ac4af-f468-43ea-ab58-d0abbdcfafa7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:30.207563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 508/1019] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:30.208787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:30.268781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:30.313678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754dcbc-d239-4863-b0b7-5cf2c1ad765b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:30.347511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0c4e2-8830-44cc-b909-bade788ef2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:30.348183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c4e2-8830-44cc-b909-bade788ef2a1 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:30.348467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c4e2-8830-44cc-b909-bade788ef2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:30.348743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0c4e2-8830-44cc-b909-bade788ef2a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:30.349446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0c4e2-8830-44cc-b909-bade788ef2a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:30.350858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0c4e2-8830-44cc-b909-bade788ef2a1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:30.475480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 512/1020] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:30 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:30.625425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d892-06fe-4ef1-b81f-f1983335bf47 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:30.625713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d892-06fe-4ef1-b81f-f1983335bf47 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:30.625993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d892-06fe-4ef1-b81f-f1983335bf47 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:30.627402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 509/1021] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:30.632886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:30.633184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:30.633471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:30.650411 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:30.652265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:30.725933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 513/1022] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:30.749855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d892-06fe-4ef1-b81f-f1983335bf47 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:30.856980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d892-06fe-4ef1-b81f-f1983335bf47 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:30.859382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:30.892791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:30.893582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:30.893903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:30.894573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:30.895602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:30.897475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:30.910310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:30.911291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:30.912814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 510/1023] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Jun 16 17:48:30 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:30.919986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:30.956320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:31.021757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15e554-1a06-4598-b458-e47ecbb6b8b2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:31.065524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:31.066333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.066624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.066908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.067645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] RBAC: Adding query filter params (name=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:31.067960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:31.069399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:31.080066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 514/1024] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:31.081708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:31.188239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c876ad-1806-438b-bc43-a3c661988927 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:31.224009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:31.224679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.224963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.225272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.225976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:31.226284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:31.227671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:31.237914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 511/1025] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:31.239683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:31.346201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:31.349672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac0b82-04b0-486e-9b3a-db75c7d87231 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:31.384614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:31.385285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.385571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.386476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.387261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] RBAC: Adding query filter params (interface=public, service_id=6db73a6718ef48fdba7132e516eb3bee, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:31.387605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:31.388965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eb1d9-d2c6-44e5-875d-f049ce3361c5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:31.397968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 515/1026] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:48:31 2022] GET /identity/v3/endpoints?service_id=6db73a6718ef48fdba7132e516eb3bee&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:31.400148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:31.432266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:31.521756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1581f77-bfef-4e9d-a0c3-380a963ef557 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:31.535251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d618c-ee08-4541-9834-2954a3ad6dba None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.535630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d618c-ee08-4541-9834-2954a3ad6dba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.535880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d618c-ee08-4541-9834-2954a3ad6dba None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.537286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 516/1027] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:31.543430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.543722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.544010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.561495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:31.562161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.562447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.562722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.562830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:31.563421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:31.564051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:31.564781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:31.572566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 512/1028] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:48:31 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:48:31.641990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 517/1029] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:31.665522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:31.772060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:31.778298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c75463-4581-4e9c-b735-4f73e0056713 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:31.813404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:31.814214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.814498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.814775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.825134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:31.833237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:31.834253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:31.835859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 513/1030] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jun 16 17:48:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:31.840711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:31.937875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:31.945154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b02548-fcde-4012-9d75-ec4ee0f5e66b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:31.980198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:31.980851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:31.981120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:31.981406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:31.982154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:31.982461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:31.983875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:31.997281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 518/1031] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:48:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:31.998233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:32.101843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e375d97-d801-4a42-974d-124589dfef79 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:32.136020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:32.136814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.137149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.137427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.148379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:32.149809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:32.167679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:32.168619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:32.174418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:32.174418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:32.174418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:32.174418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:32.174418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 514/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:32.175893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:32.288065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0bd0b-cf3d-4c8a-b234-32a9038986d3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:32.323721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:32.324387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.324669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.324940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.325656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:32.325962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:32.327313 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:32.340113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 519/1033] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:32.345514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:32.357729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:32.447470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc50d6-85c0-4a1f-b09e-403a43b6bb6f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:32.481361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:32.482145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.482430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.482707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.490183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:32.491631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:32.505379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:32.506346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:32.507331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:32.511873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 515/1034] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:48:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:32.513162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:32.532519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:32.616122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ce889-ded8-4886-a67c-545f7d837ac2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:32.650837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:32.652120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.652402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.652672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.653523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:32.653832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:32.655320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937520c-7b81-48ff-99fd-8bea5dd06209 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:32.681123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 520/1035] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:48:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:32.682778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:32.728761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d88e9-2202-4d6f-b858-3f46a10f4929 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.729089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d88e9-2202-4d6f-b858-3f46a10f4929 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.729378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d88e9-2202-4d6f-b858-3f46a10f4929 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.731098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 521/1036] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:32.736029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.736315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.736586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.754062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:32.755244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:32.756746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:32.793458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c5c3f-bf67-4a8b-b55c-86323a665f65 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:32.825234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 522/1037] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:32.831334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:32.832014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.832301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.832582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.849061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:32.852771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:32.908544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:32.909991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:32.951954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac76f44-90b2-45f2-add1-37afa3faac6e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:32.979062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:32.985484 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:32.986280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:32.986564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:32.986839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:32.988067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:32.989495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:32.991283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 523/1038] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:32.997362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.017968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 516/1039] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:48:33.023290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.076575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:33.107049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74e917-f673-4469-9a39-e825c01d97a1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:33.130016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ce989-b9e5-4bca-8633-314f40df4ce8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:33.145045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:33.145813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.146169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.146519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.147303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:33.148812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:33.165286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:33.172047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.172047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.172047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.178994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:33.180590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:33.199659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:33.200823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:33.201941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:33.201941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:33.201941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:33.201941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:33.203526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 517/1040] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:33.206317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.227808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:33.277990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 524/1041] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:48:32 2022] POST /identity/v3/users => generated 285 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:48:33.320129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b43655-5765-4de2-924f-4eefc3e35470 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:33.358016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:33.358761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.359050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.359325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.360089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:33.360398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:33.361772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:33.375697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 518/1042] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:33.381262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.494432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0bd6-880a-4e26-b982-5084c27bbc09 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:33.523425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:33.537858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:33.537858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.537858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.537858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.543751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:33.545219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:33.560946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:33.561930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:33.562992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 525/1043] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:33.568916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.656536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:33.676504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e52c2-a77d-4d68-81a9-5023c4473a37 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:33.715806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:33.716474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.716755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.717029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.717812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:33.718118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:33.719568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e90bb-557a-4c0a-9b5c-d7ee8aac0c5c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:33.737525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 519/1044] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:33.739130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.757753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd7e2-57cc-46f3-9302-ccda3fbd8ffd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.758041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd7e2-57cc-46f3-9302-ccda3fbd8ffd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.758314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd7e2-57cc-46f3-9302-ccda3fbd8ffd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.763449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 520/1045] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:48:33.764919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b807da-a404-47bb-99c9-26613607502b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.765195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b807da-a404-47bb-99c9-26613607502b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.765807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b807da-a404-47bb-99c9-26613607502b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.784729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c3b807da-a404-47bb-99c9-26613607502b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:33.785934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c3b807da-a404-47bb-99c9-26613607502b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:33.855347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce14d16-316c-4958-8560-26216c8e1ae7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:33.862346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 521/1046] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:33.885045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b807da-a404-47bb-99c9-26613607502b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:33.892648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:33.893323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:33.893995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:33.894275 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:33.895107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] RBAC: Adding query filter params (role.id=788818f843d94f6e93bc573dc74f60c8, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=9279195b1ed64901a63a3fce8d0bb610) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:33.895800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:33.897559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:33.905327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 526/1047] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:33 2022] GET /identity/v3/role_assignments?user.id=9279195b1ed64901a63a3fce8d0bb610&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=788818f843d94f6e93bc573dc74f60c8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:33.961942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:33.994995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b807da-a404-47bb-99c9-26613607502b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:34.032073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:34.032972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:34.033319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:34.033819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:34.034547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:34.036111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:34.050258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:34.051770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:48:34.052837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:34.053739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 522/1048] 10.222.0.58 () {56 vars in 1105 bytes} [Thu Jun 16 17:48:33 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:34.059425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:34.123044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:34.172694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d54dae-1ccb-4962-b8de-8c577f4619b4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:34.207136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:34.207830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:34.208159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:34.208438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:34.209139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] RBAC: Adding query filter params (name=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:34.209447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:34.210799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:34.220774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 527/1049] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:34.221909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:34.325268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd70e2f-7bef-4ad1-9862-630dd127e47c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:34.358614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:34.359272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:34.359582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:34.359855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:34.360552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:34.360854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:34.362200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197fe3d-be05-4b2a-914a-0a2f092495c5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:34.363453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:34.380536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 523/1050] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:34.380705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:34.475508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ff35-e853-4dda-9792-800006208c55 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:34.495652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:34.515656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddf430-fb87-4c78-a493-1bfa994b2396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:34.516319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf430-fb87-4c78-a493-1bfa994b2396 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:34.516599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf430-fb87-4c78-a493-1bfa994b2396 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:34.516878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddf430-fb87-4c78-a493-1bfa994b2396 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:34.517626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddf430-fb87-4c78-a493-1bfa994b2396 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:34.518979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddf430-fb87-4c78-a493-1bfa994b2396 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:34.622883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 528/1051] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:48:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:35.545087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956af08-88a9-47e8-8056-e2666eb72c2d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:35.545605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956af08-88a9-47e8-8056-e2666eb72c2d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:35.545862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956af08-88a9-47e8-8056-e2666eb72c2d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:35.548312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 524/1052] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:35.553869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:35.554289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:35.554476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:35.571756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:35.573479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:35.655870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 529/1053] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:35.679454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956af08-88a9-47e8-8056-e2666eb72c2d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:35.786109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956af08-88a9-47e8-8056-e2666eb72c2d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:35.820265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:35.820981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:35.821264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:35.821541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:35.832911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:35.840605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:35.841543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:35.842440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:35.846817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 525/1054] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:35.848088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:35.971462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:35.975616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a5619-3f2f-4f63-9fdc-0dd0b67396df None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.012536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.013244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.013531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.013810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.014582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:36.014970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.016409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f384b31-4d5c-4b33-862e-48fd7c803c15 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.031345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 530/1055] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:48:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:36.033083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.070911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584da964-1a3d-45cc-9469-633b5cbd6cb2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.071195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584da964-1a3d-45cc-9469-633b5cbd6cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.071443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584da964-1a3d-45cc-9469-633b5cbd6cb2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.072883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 531/1056] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:48:36.079110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.079501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.079794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.098125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:36.098511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.099485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:36.142436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f57b4d-8a12-4104-a993-e0832a3eedf6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.173667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 532/1057] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:36.178499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.179389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.179678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.179952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] PATH_INFO: `/v3/users/fdf1575a8bf04630b397d3a92021ae7f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.197576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.199797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] RBAC: Authorizing `identity:get_user(user_id=fdf1575a8bf04630b397d3a92021ae7f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.201251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.203339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 526/1058] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/users/fdf1575a8bf04630b397d3a92021ae7f => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:36.208993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.300172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.310746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e44e7-a1cd-4b3a-9da3-9ea329b97bf8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.335596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da9fb-74c0-4de9-801f-f2bde2fdef6f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.348020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.348852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.349140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.349421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.353953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.361416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:36.370452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:36.371571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:36.372442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 533/1059] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:36.375962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.376697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.376984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.377300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.377663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.454052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.455619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.479907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47879f93-c21b-4957-99e6-762e85e301f8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.514370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.515050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.515335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.515644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.516389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:36.516702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.518068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.527133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 534/1060] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:36.532396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.548205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.566811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 527/1061] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:48:36.573050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.591197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.637976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efd43e-296e-4e31-83f9-d5985523fd34 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.676623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.677423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.677708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.677987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.681967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3cd6b-8dae-43d2-9598-a9cbe9e1257d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.689044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.690493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:36.708530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:36.709608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:36.711197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:36.711197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:36.711197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:36.711197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:36.711197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 535/1062] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:36.716025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.721504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.722384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.722671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.722949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.724264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] RBAC: Authorizing `identity:get_project(project_id=8793d908cc7e43b099aa580ddde21cb4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.725748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.727612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 528/1063] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:36.734489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.787784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.824926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:36.828800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0a428-c4b6-43e7-95e0-ba09665b2fa8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.841431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276b468-c10f-4823-8d6e-cd3cd4f1ae44 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:36.863655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.864320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.864603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.864876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.865586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:36.865894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.867242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460f7e-cbf7-4f40-b22c-5d3d307bea16 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.875858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:36.876523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.876965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.877611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.878441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=8793d908cc7e43b099aa580ddde21cb4, user.id=fdf1575a8bf04630b397d3a92021ae7f) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:36.878758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:36.879207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 536/1064] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:36.883935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ad821-b1c5-432b-919e-224cd8b04db0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:36.884032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62620910-7074-48f7-a423-0297dc2458eb None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.884032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62620910-7074-48f7-a423-0297dc2458eb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.884032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62620910-7074-48f7-a423-0297dc2458eb None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.884032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 537/1065] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:36.887764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62620910-7074-48f7-a423-0297dc2458eb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:36.893377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 529/1066] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/role_assignments?user.id=fdf1575a8bf04630b397d3a92021ae7f&scope.project.id=8793d908cc7e43b099aa580ddde21cb4&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:36.893377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:36.893377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:36.893377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:36.913433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:36.914697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:37.010014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 530/1067] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:37.017943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62620910-7074-48f7-a423-0297dc2458eb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.034688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.048541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.062237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.063079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.063402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.063735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.071821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.073859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:37.090304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:37.091359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:37.092407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 538/1068] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:48:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:37.097464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.128958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.152249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c853a-0d37-4b9b-a1a6-bd0615eb057f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.189210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.189871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.190159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.190440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.217878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcc1bb-742e-462c-9b41-33ad31424955 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.264468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.265158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.265445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.265725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.266575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:37.266890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.268367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.285972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.286758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 539/1069] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:37.288464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.292383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.338070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.382765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.399692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdf6e9-afaa-482b-8103-480a27acc19f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.408365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 531/1070] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:37.413999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.438006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.438808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.439097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.439391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.451156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.452854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:37.472939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:37.473863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:37.475512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:37.475512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:37.475512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:37.475512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:37.475512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 540/1071] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:37.479862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.523790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bc18b-e3a1-4c74-80f2-062c30a6f64b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.564653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.565453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.565733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.566008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.579187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.580294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.581722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.589434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9026f-8316-4866-8c8a-69f425169775 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:37.599651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:37.601441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:37.605041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:37.605041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:37.605041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:37.605041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:37.605041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 532/1072] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:37.607605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.634306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.634895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.635220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.635535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.636285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:37.636589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.637946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.641072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.655664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 541/1073] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:37.656726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.718725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0c63d-8f39-4d2f-9701-689dd8008965 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.754743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.755433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.755711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.755989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.756705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:37.757015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.758377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.763799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a5c51-ec4c-438e-9985-07e5413d0e95 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.769907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 533/1074] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:37.801593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.802415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.802705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.802978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.811439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.812859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:37.827963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:37.828938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:37.834870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 542/1075] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:37.835395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:37.939692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:37.949795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7deda-6879-4603-93ee-a4ce491d00cc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:37.986384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:37.987052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:37.987339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:37.987646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:37.988447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:37.988745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:37.990162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623407cc-80bd-4f24-bf8c-934cedad7a9d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:38.003223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 534/1076] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:38.009391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:38.029094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:38.116558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827ba91-a01d-43fc-9963-077cac116383 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:38.149261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb302e63-326e-46ad-a2c7-98bba3008b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:38.150139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb302e63-326e-46ad-a2c7-98bba3008b77 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:38.150418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb302e63-326e-46ad-a2c7-98bba3008b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:38.150691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb302e63-326e-46ad-a2c7-98bba3008b77 None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/9279195b1ed64901a63a3fce8d0bb610/roles/788818f843d94f6e93bc573dc74f60c8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:38.153715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb302e63-326e-46ad-a2c7-98bba3008b77 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=9279195b1ed64901a63a3fce8d0bb610, role_id=788818f843d94f6e93bc573dc74f60c8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:38.155135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb302e63-326e-46ad-a2c7-98bba3008b77 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:38.237353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 543/1077] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:48:38 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/9279195b1ed64901a63a3fce8d0bb610/roles/788818f843d94f6e93bc573dc74f60c8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:48:39.065672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ae2d-2d8d-4b09-984f-fd72627bd8df None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.066208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ae2d-2d8d-4b09-984f-fd72627bd8df None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.066208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265ae2d-2d8d-4b09-984f-fd72627bd8df None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.067638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 535/1078] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:39.073210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.073626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.073759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.090681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:39.091914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:39.159100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 544/1079] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:39.181403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ae2d-2d8d-4b09-984f-fd72627bd8df None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:39.283906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265ae2d-2d8d-4b09-984f-fd72627bd8df None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:39.319112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:39.319847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.320136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.320414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.331109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:39.339225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:39.340269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:39.341147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 536/1080] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:39.346518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:39.350264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:39.445387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedd2e7-1160-41b6-a49f-e1c65b0bae0d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:39.479716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:39.480388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.480673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.480949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.481687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:39.481996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:39.483358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:39.491463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 545/1081] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:39.497829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:39.511999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:39.597245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06505867-b3b0-4c80-ae0d-067529c79d34 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:39.643598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:39.644474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.644760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.645038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] PATH_INFO: `/v3/users/fdf1575a8bf04630b397d3a92021ae7f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.646236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] RBAC: Authorizing `identity:get_user(user_id=fdf1575a8bf04630b397d3a92021ae7f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:39.647695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:39.649486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 537/1082] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:39 2022] GET /identity/v3/users/fdf1575a8bf04630b397d3a92021ae7f => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:39.655266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:39.719486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:39.756790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e140e-5219-4ccf-8b6b-30a6ca8d3aa3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:39.790465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1907c32-b2be-4877-8b43-ec6cade81fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:39.791420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1907c32-b2be-4877-8b43-ec6cade81fff None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.791746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1907c32-b2be-4877-8b43-ec6cade81fff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.792099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1907c32-b2be-4877-8b43-ec6cade81fff None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.793214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1907c32-b2be-4877-8b43-ec6cade81fff None admin] RBAC: Authorizing `identity:get_project(project_id=8793d908cc7e43b099aa580ddde21cb4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:39.794647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1907c32-b2be-4877-8b43-ec6cade81fff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:39.796661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 546/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:48:39 2022] GET /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:39.802586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:39.880717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:39.901100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77cee3-cf2b-4951-b2b4-10e984aa3bae None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:39.935293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe388d-2791-457f-8dfb-3adb8ebf5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:39.936205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe388d-2791-457f-8dfb-3adb8ebf5e87 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:39.936494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe388d-2791-457f-8dfb-3adb8ebf5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:39.936773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe388d-2791-457f-8dfb-3adb8ebf5e87 None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4/users/fdf1575a8bf04630b397d3a92021ae7f/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:39.938779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe388d-2791-457f-8dfb-3adb8ebf5e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=8793d908cc7e43b099aa580ddde21cb4, user_id=fdf1575a8bf04630b397d3a92021ae7f, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:39.940226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe388d-2791-457f-8dfb-3adb8ebf5e87 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:40.009189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 538/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:48:39 2022] PUT /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4/users/fdf1575a8bf04630b397d3a92021ae7f/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:48:40.411910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516edb2d-78ce-41b7-a5e8-2bde2badc035 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:40.412435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516edb2d-78ce-41b7-a5e8-2bde2badc035 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:40.412549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516edb2d-78ce-41b7-a5e8-2bde2badc035 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:40.413836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 547/1085] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:40.419468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:40.419896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:40.420023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:40.437822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:40.439000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:40.506272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 539/1086] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:40.530753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516edb2d-78ce-41b7-a5e8-2bde2badc035 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:40.633102 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516edb2d-78ce-41b7-a5e8-2bde2badc035 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:40.666052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:40.667920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:40.668202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:40.668478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:40.670407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:40.679836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:40.690420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:40.691520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:40.692391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 548/1087] 10.222.0.58 () {56 vars in 1120 bytes} [Thu Jun 16 17:48:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:40.699182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:40.772969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:40.808149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192c68f-b579-4496-b2e0-d1cf103fe95d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:40.841918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:40.842573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:40.842852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:40.843120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:40.843877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:40.844180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:40.845515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:40.853164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 540/1088] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:48:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:40.859410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:40.956173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5a4ce-7b5a-4704-8135-ec503ce41162 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:40.990020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:40.990794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:40.991076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:40.991350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:41.001428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:41.002906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:41.020330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:41.021880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:41.022773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:41.022773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:41.022773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:41.022773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:41.022773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 549/1089] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:41.028028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:41.132758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298531f-364a-4dae-97f2-967249a64702 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:41.160857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:41.166310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:41.166977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:41.167261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:41.167564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:41.168285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:41.168596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:41.169959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:41.182268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 541/1090] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:41.187972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:41.236644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:41.294251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bf796-9d91-4395-a88b-0eb9323d6148 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:41.328779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:41.329562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:41.329848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:41.330120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:41.337384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:41.339200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:48:41.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:41.354178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:41.360267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 550/1091] 10.222.0.58 () {56 vars in 1093 bytes} [Thu Jun 16 17:48:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:41.361120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:41.458322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225e133-b48b-4129-8d30-0367ceb2a437 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:41.492914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:41.493569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:41.493847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:41.494118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:41.495939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:41.496251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:41.497652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:41.514693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 542/1092] 10.222.0.58 () {56 vars in 1133 bytes} [Thu Jun 16 17:48:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:41.520315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:41.626536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a71f8e-e9d8-40b0-ba3d-fb1ca930c31e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:41.633452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:41.663568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:41.664266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:41.664566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:41.665130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:41.734830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:41.736310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:41.833463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 551/1093] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:41.839824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:41.888369 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:41.938238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ef2b0-7482-4cf4-bf4c-78e57b469dd9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:41.971619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:41.972410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:41.972689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:41.972961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:41.983457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:41.984885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-97ac98d6-7e34-4e95-a2f8-70b6f3ad030f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:48:42.002073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:42.003008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:42.003943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:48:42.003943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:48:42.003943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:48:42.003943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:42.003943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 543/1094] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:48:42.009074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.017645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:42.116384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef313e86-47b5-4550-bc93-b666f406b48c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.150544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.151166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.151483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.151761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.152481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:42.152789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.154154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.155526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e395636-4ddb-49d3-89dc-94e47fb1f1e4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.155816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e395636-4ddb-49d3-89dc-94e47fb1f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.156092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e395636-4ddb-49d3-89dc-94e47fb1f1e4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.157495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 544/1095] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:48:42.162416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.162697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.162970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.165779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 552/1096] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:42.172096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.180651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:42.182401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:42.237538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:42.254128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 545/1097] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:48:42.273899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65088f82-3114-4202-a11a-3bb0f886b398 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.278595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.308508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.309423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.309705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.309830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:42.309982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.316948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.318367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:48:42.332267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:42.333168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:48:42.334232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 553/1098] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:42.339273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.382013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ec87c-bd0c-4e86-966c-37d0d176f030 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.415267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.416680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.417159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.417602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.428022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:48:42.436420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:48:42.437534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:48:42.438393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 546/1099] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:48:42.443548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.445016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbe47e-2d10-4809-9740-38f23882ca9c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.471139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:42.480586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.481238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.481521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.481797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.482577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:42.482882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.484321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.496014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 554/1100] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:42.501906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.537583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:42.548406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f314d0e-ba30-42d4-9ad7-b2628455d0e1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.582250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.582909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.583188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.583494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.584226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:42.584528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.585879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.595196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 547/1101] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:42.599801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6c300-e41d-4942-a535-b9ff310241eb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.600929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.634000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.634657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.634989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.635405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.636368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] RBAC: Adding query filter params (role.id=788818f843d94f6e93bc573dc74f60c8, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=9279195b1ed64901a63a3fce8d0bb610) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:42.636684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.638110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.643943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 555/1102] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/role_assignments?user.id=9279195b1ed64901a63a3fce8d0bb610&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=788818f843d94f6e93bc573dc74f60c8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:42.705447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae5a3-a3db-4170-8f53-3601336afd36 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.743505 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.744597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.744909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.745198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] PATH_INFO: `/v3/users/fdf1575a8bf04630b397d3a92021ae7f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.746469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] RBAC: Authorizing `identity:get_user(user_id=fdf1575a8bf04630b397d3a92021ae7f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.748010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:42.749887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 548/1103] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/users/fdf1575a8bf04630b397d3a92021ae7f => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:42.755598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:42.813015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:42.863777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba559a-3330-407e-91cc-c3ff33124e7c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:42.898036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:42.898694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:42.898978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:42.899249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:42.970350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:42.971756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:43.021040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:43.074361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 556/1104] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:43.080398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:43.107018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:43.187510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd86b78-2cc3-4b74-a6d7-9548de6f7729 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:43.223465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de9ade-35d0-4bf6-bfd2-b6ec8a1b65e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:43.224248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de9ade-35d0-4bf6-bfd2-b6ec8a1b65e6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:43.224530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de9ade-35d0-4bf6-bfd2-b6ec8a1b65e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:43.224808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de9ade-35d0-4bf6-bfd2-b6ec8a1b65e6 None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:43.225982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de9ade-35d0-4bf6-bfd2-b6ec8a1b65e6 None admin] RBAC: Authorizing `identity:get_project(project_id=8793d908cc7e43b099aa580ddde21cb4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:43.227421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de9ade-35d0-4bf6-bfd2-b6ec8a1b65e6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:43.229685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 549/1105] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:48:43 2022] GET /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:43.235887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:43.352332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662d0ab-92c6-4f42-b2f4-debaacb9f673 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:43.389765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:43.390438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:43.390725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:43.391008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:43.392020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=8793d908cc7e43b099aa580ddde21cb4, user.id=fdf1575a8bf04630b397d3a92021ae7f) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:43.392217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:43.393613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ea1a3-aa5c-49f5-992a-6f78b6615ba8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:43.400418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 557/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:43 2022] GET /identity/v3/role_assignments?user.id=fdf1575a8bf04630b397d3a92021ae7f&scope.project.id=8793d908cc7e43b099aa580ddde21cb4&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:45.540546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1d21-2c2c-4640-add3-17f164af8630 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:45.541119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1d21-2c2c-4640-add3-17f164af8630 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:45.541119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1d21-2c2c-4640-add3-17f164af8630 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:45.542969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 550/1107] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:45.548319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:45.548760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:45.548884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:45.565165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:45.566356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:45.575291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:45.631551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 558/1108] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:45.656882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f1d21-2c2c-4640-add3-17f164af8630 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:45.743647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:45.756614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f1d21-2c2c-4640-add3-17f164af8630 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:45.789472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474276ed-3a5c-4642-913c-1be99898f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:45.790247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474276ed-3a5c-4642-913c-1be99898f006 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:45.790529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474276ed-3a5c-4642-913c-1be99898f006 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:45.790901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474276ed-3a5c-4642-913c-1be99898f006 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:45.792112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474276ed-3a5c-4642-913c-1be99898f006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:45.793519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474276ed-3a5c-4642-913c-1be99898f006 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:45.795113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 551/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:45.801102 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:45.897151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a816c-3878-46e3-b8b0-3c4ea39759d1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:45.930144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c86193-fc7e-4e65-a581-0bd59965946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:45.930803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86193-fc7e-4e65-a581-0bd59965946c None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:45.931088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86193-fc7e-4e65-a581-0bd59965946c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:45.931429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c86193-fc7e-4e65-a581-0bd59965946c None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:45.932659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c86193-fc7e-4e65-a581-0bd59965946c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:45.934067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c86193-fc7e-4e65-a581-0bd59965946c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:46.026440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:46.031385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 559/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:48:45 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:48:48.199984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664daba3-9acb-42a3-9ce9-3687c6fca1e4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:48.200554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664daba3-9acb-42a3-9ce9-3687c6fca1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:48.200554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664daba3-9acb-42a3-9ce9-3687c6fca1e4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:48.201897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 552/1111] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:48.207012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:48.207424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:48.207596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:48.224596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:48.225787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:48.290681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 560/1112] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:48.314184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664daba3-9acb-42a3-9ce9-3687c6fca1e4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:48.418787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664daba3-9acb-42a3-9ce9-3687c6fca1e4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:48.451724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:48.452508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:48.452793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:48.453068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] PATH_INFO: `/v3/roles/89845870f208412e84a72ff363bae771` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:48.454523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] RBAC: Authorizing `identity:get_role(role_id=89845870f208412e84a72ff363bae771)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:48.455910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:48.457259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 553/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:48 2022] GET /identity/v3/roles/89845870f208412e84a72ff363bae771 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:48.463729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:48.560567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661d6ab-9866-4331-a39e-e81073127c60 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:48.588098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:48.593669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:48.594532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:48.594822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:48.595095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] PATH_INFO: `/v3/users/bf883f27265043f284913bbb0adeec6c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:48.607872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] RBAC: Authorizing `identity:get_user(user_id=bf883f27265043f284913bbb0adeec6c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:48.609293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:48.610930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 561/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:48 2022] GET /identity/v3/users/bf883f27265043f284913bbb0adeec6c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:48.616774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:48.643374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:48.716125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe74b6-f56c-4761-8ec9-e64bc9e3c16b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:48.750152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:48.750792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:48.751077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:48.751355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:48.816082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:48.817512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:48.840744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:48.913191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 554/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:48.919810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:49.016834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b865d7-f54d-429f-8e6a-c76ac53c1ba6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:49.049886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63128e70-e6db-4b29-a86c-75fc512c332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:49.050734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128e70-e6db-4b29-a86c-75fc512c332a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:49.051049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128e70-e6db-4b29-a86c-75fc512c332a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:49.051326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63128e70-e6db-4b29-a86c-75fc512c332a None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:49.052608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128e70-e6db-4b29-a86c-75fc512c332a None admin] RBAC: Authorizing `identity:get_project(project_id=8793d908cc7e43b099aa580ddde21cb4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:49.054026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63128e70-e6db-4b29-a86c-75fc512c332a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:49.055748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 562/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:48:48 2022] GET /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:49.062577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:49.157775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dffcea-a899-4326-bfa2-40fb5f8e9ec9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:49.181703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:49.192027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:49.192674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:49.192961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:49.193236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:49.194057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] RBAC: Adding query filter params (role.id=89845870f208412e84a72ff363bae771, scope.project.id=8793d908cc7e43b099aa580ddde21cb4, user.id=bf883f27265043f284913bbb0adeec6c) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:49.194368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:49.195819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441d6b4-c9c8-4db7-b98d-db4058290cb1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:49.202108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 555/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:49 2022] GET /identity/v3/role_assignments?user.id=bf883f27265043f284913bbb0adeec6c&scope.project.id=8793d908cc7e43b099aa580ddde21cb4&role.id=89845870f208412e84a72ff363bae771 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:51.376181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a02ac3-8bdd-4698-b310-89620679d9e7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:51.376713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a02ac3-8bdd-4698-b310-89620679d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:51.376713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a02ac3-8bdd-4698-b310-89620679d9e7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:51.378080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 563/1118] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:51.383410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:51.383856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:51.384481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:51.402504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:51.403723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:51.468054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 556/1119] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:51.491680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a02ac3-8bdd-4698-b310-89620679d9e7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:51.548744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:51.592512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a02ac3-8bdd-4698-b310-89620679d9e7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:51.625231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:51.626073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:51.626354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:51.626626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] PATH_INFO: `/v3/roles/89845870f208412e84a72ff363bae771` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:51.628096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] RBAC: Authorizing `identity:get_role(role_id=89845870f208412e84a72ff363bae771)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:51.629444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:51.630791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 564/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:51 2022] GET /identity/v3/roles/89845870f208412e84a72ff363bae771 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:48:51.637692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:51.733049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6b457-99ab-441f-929f-f87f25d6e6a1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:51.749942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:51.766819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:51.767752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:51.768035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:51.768310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] PATH_INFO: `/v3/users/bf883f27265043f284913bbb0adeec6c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:51.769498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] RBAC: Authorizing `identity:get_user(user_id=bf883f27265043f284913bbb0adeec6c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:51.770961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:51.772600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 557/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:51 2022] GET /identity/v3/users/bf883f27265043f284913bbb0adeec6c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:51.779526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:51.879405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21563ed9-70ab-4e28-bef0-0c5b5db38989 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:51.902127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:51.912326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706396aa-a780-4b81-bc66-e55667fa7ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:51.913174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706396aa-a780-4b81-bc66-e55667fa7ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:51.913462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706396aa-a780-4b81-bc66-e55667fa7ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:51.913742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706396aa-a780-4b81-bc66-e55667fa7ce4 None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:51.914915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706396aa-a780-4b81-bc66-e55667fa7ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=8793d908cc7e43b099aa580ddde21cb4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:51.917045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706396aa-a780-4b81-bc66-e55667fa7ce4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:51.918729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 565/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:48:51 2022] GET /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:51.925016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:52.022995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580c89-e165-4a71-bde4-e5af0da85882 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:52.057550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3994f65-1437-4479-8d2d-20a3a787f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:52.058419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3994f65-1437-4479-8d2d-20a3a787f4d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:52.058698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3994f65-1437-4479-8d2d-20a3a787f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:52.058975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3994f65-1437-4479-8d2d-20a3a787f4d4 None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4/users/bf883f27265043f284913bbb0adeec6c/roles/89845870f208412e84a72ff363bae771` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:52.061167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3994f65-1437-4479-8d2d-20a3a787f4d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8793d908cc7e43b099aa580ddde21cb4, user_id=bf883f27265043f284913bbb0adeec6c, role_id=89845870f208412e84a72ff363bae771)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:52.062566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3994f65-1437-4479-8d2d-20a3a787f4d4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:52.160610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 558/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:48:51 2022] PUT /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4/users/bf883f27265043f284913bbb0adeec6c/roles/89845870f208412e84a72ff363bae771 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:48:54.292659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07809f7-9b53-4a24-ba17-8d4718171e41 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:54.293209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07809f7-9b53-4a24-ba17-8d4718171e41 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:54.293303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07809f7-9b53-4a24-ba17-8d4718171e41 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:54.294638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 566/1124] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:54.299975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:54.300380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:54.300538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:54.317595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:54.318785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:54.360387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:54.385598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 559/1125] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:54.408878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07809f7-9b53-4a24-ba17-8d4718171e41 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:54.511925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07809f7-9b53-4a24-ba17-8d4718171e41 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:54.546310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:54.547163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:54.547476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:54.547759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] PATH_INFO: `/v3/roles/89845870f208412e84a72ff363bae771` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:54.549240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] RBAC: Authorizing `identity:get_role(role_id=89845870f208412e84a72ff363bae771)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:54.550600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:54.552018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 567/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:54 2022] GET /identity/v3/roles/89845870f208412e84a72ff363bae771 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:54.558060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:54.653822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d21190-1ed8-4764-a9b4-d2432cd518ad None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:54.688079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:54.688931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:54.689215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:54.689492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] PATH_INFO: `/v3/users/bf883f27265043f284913bbb0adeec6c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:54.690669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] RBAC: Authorizing `identity:get_user(user_id=bf883f27265043f284913bbb0adeec6c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:54.692124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:54.693748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 560/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:48:54 2022] GET /identity/v3/users/bf883f27265043f284913bbb0adeec6c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:54.699464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:54.756719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:54.796430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e6fd4-08bd-4412-bdb2-3da039beed80 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:54.830072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:54.830714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:54.830998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:54.831275 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:54.896023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:54.897454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:54.952317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:54.992684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 568/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:48:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:48:54.998154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:55.043176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:55.095812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d66b7-ba4e-4808-b917-a319659f27d7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:55.128066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7536-7833-4f13-856a-e02791210981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:55.128906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7536-7833-4f13-856a-e02791210981 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:55.129190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7536-7833-4f13-856a-e02791210981 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:55.129470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7536-7833-4f13-856a-e02791210981 None admin] PATH_INFO: `/v3/projects/8793d908cc7e43b099aa580ddde21cb4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:55.130625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7536-7833-4f13-856a-e02791210981 None admin] RBAC: Authorizing `identity:get_project(project_id=8793d908cc7e43b099aa580ddde21cb4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:55.132063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7536-7833-4f13-856a-e02791210981 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:55.134227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 561/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:48:54 2022] GET /identity/v3/projects/8793d908cc7e43b099aa580ddde21cb4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:55.140314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:55.236989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d55fc7-772e-46cc-80aa-c92a5df068e0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:55.272386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:55.273029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:55.273310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:55.273586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:55.274903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] RBAC: Adding query filter params (role.id=89845870f208412e84a72ff363bae771, scope.project.id=8793d908cc7e43b099aa580ddde21cb4, user.id=bf883f27265043f284913bbb0adeec6c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:48:55.275221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:55.276666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69322f-c20e-4ade-8507-7f76488df3e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:55.282328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 569/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:48:55 2022] GET /identity/v3/role_assignments?user.id=bf883f27265043f284913bbb0adeec6c&scope.project.id=8793d908cc7e43b099aa580ddde21cb4&role.id=89845870f208412e84a72ff363bae771 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:57.474826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4782a-206b-49ed-80b5-4aaaf5519d6f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:57.475350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4782a-206b-49ed-80b5-4aaaf5519d6f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:57.475350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4782a-206b-49ed-80b5-4aaaf5519d6f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:57.476744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 562/1131] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:48:57.482701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:57.483126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:57.483259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:57.499688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:48:57.500819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:48:57.517594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:57.566643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 570/1132] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:48:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:48:57.590391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4782a-206b-49ed-80b5-4aaaf5519d6f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:57.694386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4782a-206b-49ed-80b5-4aaaf5519d6f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:57.707205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:48:57.728235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390b343-b321-4e9c-836c-c010450dd700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:57.729015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b343-b321-4e9c-836c-c010450dd700 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:57.729299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b343-b321-4e9c-836c-c010450dd700 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:57.729572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390b343-b321-4e9c-836c-c010450dd700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:57.730733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390b343-b321-4e9c-836c-c010450dd700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:57.732179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390b343-b321-4e9c-836c-c010450dd700 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:57.733830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 563/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:48:57.739597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:48:57.835030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6784b-d6bf-46fc-bfb5-f47979d676db None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:48:57.868084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33062cf5-45f9-43f3-bf31-173f6babd106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:48:57.868753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062cf5-45f9-43f3-bf31-173f6babd106 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:48:57.869036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062cf5-45f9-43f3-bf31-173f6babd106 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:48:57.869341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062cf5-45f9-43f3-bf31-173f6babd106 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:48:57.870084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33062cf5-45f9-43f3-bf31-173f6babd106 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:48:57.871535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33062cf5-45f9-43f3-bf31-173f6babd106 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:48:57.991578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 571/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:48:57 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:49:00.169071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2feb6-9577-4093-98f6-f8761df73357 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:00.169616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2feb6-9577-4093-98f6-f8761df73357 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:00.169616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2feb6-9577-4093-98f6-f8761df73357 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:00.171250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 564/1135] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:00.176431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04daeec-0747-4561-8626-bf507d424753 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:00.176837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04daeec-0747-4561-8626-bf507d424753 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:00.176981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04daeec-0747-4561-8626-bf507d424753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:00.188500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:00.193778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f04daeec-0747-4561-8626-bf507d424753 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:00.194979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f04daeec-0747-4561-8626-bf507d424753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:00.260848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 572/1136] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:00.288131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2feb6-9577-4093-98f6-f8761df73357 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:00.391529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2feb6-9577-4093-98f6-f8761df73357 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:00.424424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85accd3e-230e-4d30-beb5-b524dbca9152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:00.425135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85accd3e-230e-4d30-beb5-b524dbca9152 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:00.425415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85accd3e-230e-4d30-beb5-b524dbca9152 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:00.425690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85accd3e-230e-4d30-beb5-b524dbca9152 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:00.426849 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85accd3e-230e-4d30-beb5-b524dbca9152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:00.428288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85accd3e-230e-4d30-beb5-b524dbca9152 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:00.429884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 565/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:49:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:00.435734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04daeec-0747-4561-8626-bf507d424753 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:00.533264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04daeec-0747-4561-8626-bf507d424753 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:00.566279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56805d-390c-48e8-ad3b-557a862ffd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:00.566940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56805d-390c-48e8-ad3b-557a862ffd7b None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:00.567228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56805d-390c-48e8-ad3b-557a862ffd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:00.567535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56805d-390c-48e8-ad3b-557a862ffd7b None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:00.568247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56805d-390c-48e8-ad3b-557a862ffd7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:00.569670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56805d-390c-48e8-ad3b-557a862ffd7b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:00.577928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:00.687987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 573/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:49:00 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:49:02.855514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83fecd-ebae-4c38-83cd-3c440d6f34ec None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:02.856115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83fecd-ebae-4c38-83cd-3c440d6f34ec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:02.856115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83fecd-ebae-4c38-83cd-3c440d6f34ec None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:02.857417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 566/1139] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:02.862580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:02.863033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:02.863132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:02.879707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:02.880923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:02.944855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 574/1140] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:02.968433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83fecd-ebae-4c38-83cd-3c440d6f34ec None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:03.017895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:03.068266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83fecd-ebae-4c38-83cd-3c440d6f34ec None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:03.101548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:03.103053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:03.103336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:03.103647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:03.113538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:49:03.121124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:03.122309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:49:03.123622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 567/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:03.128311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:03.224437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57d165-ac65-47a4-a614-1a73d254388c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:03.245839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:03.257103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:03.257766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:03.258692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:03.258984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:03.259761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:03.260067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:03.261424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:03.269406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 575/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:03.275829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:03.374419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35985274-cb6d-40d1-92e2-00753eb1ffda None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:03.382796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:03.408058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:03.408922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:03.409208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:03.409485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] PATH_INFO: `/v3/users/5a559db65c3d41ecb78418c117dc9185` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:03.423209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] RBAC: Authorizing `identity:get_user(user_id=5a559db65c3d41ecb78418c117dc9185)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:03.424685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:03.426319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 568/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:49:03 2022] GET /identity/v3/users/5a559db65c3d41ecb78418c117dc9185 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:03.432608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:03.532666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8738-c56e-4f4e-aeee-1ddfd761780b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:03.565626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:03.566266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:03.566549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:03.566824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:03.616597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:03.632065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:03.634526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:03.731149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 576/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:49:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:49:03.737014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:03.833545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927595b-54b0-4d76-9701-cc732c60fcab None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:03.867171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc173d-65b5-4e8c-829a-bd38feea841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:03.868582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc173d-65b5-4e8c-829a-bd38feea841b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:03.868864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc173d-65b5-4e8c-829a-bd38feea841b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:03.869138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc173d-65b5-4e8c-829a-bd38feea841b None admin] PATH_INFO: `/v3/projects/00bbb8057bbf48af917aff638bc581f8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:03.870311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc173d-65b5-4e8c-829a-bd38feea841b None admin] RBAC: Authorizing `identity:get_project(project_id=00bbb8057bbf48af917aff638bc581f8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:03.871786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc173d-65b5-4e8c-829a-bd38feea841b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:03.873450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 569/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:49:03 2022] GET /identity/v3/projects/00bbb8057bbf48af917aff638bc581f8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:03.879596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:03.964855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:03.980605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729710bf-0b56-4e17-bc58-51fffda83800 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:04.014524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:04.015173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:04.015539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:04.015819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:04.016640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=00bbb8057bbf48af917aff638bc581f8, user.id=5a559db65c3d41ecb78418c117dc9185) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:04.016952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:04.018374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631eebe-027d-40ec-b938-b9d971b8f2b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:04.023831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 577/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:49:03 2022] GET /identity/v3/role_assignments?user.id=5a559db65c3d41ecb78418c117dc9185&scope.project.id=00bbb8057bbf48af917aff638bc581f8&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:06.118410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae05a6-0b45-4cd9-8d30-3b58b3e885eb None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.118961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae05a6-0b45-4cd9-8d30-3b58b3e885eb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.118961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae05a6-0b45-4cd9-8d30-3b58b3e885eb None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.120357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 570/1147] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:06.125656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.126082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.126188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.143562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:06.144744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:06.208407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 578/1148] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:06.231173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae05a6-0b45-4cd9-8d30-3b58b3e885eb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:06.265346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:06.330616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae05a6-0b45-4cd9-8d30-3b58b3e885eb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:06.363569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:06.364352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.364636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.364911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.374873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:49:06.382334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:06.383911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:49:06.384813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 571/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:49:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:06.390301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:06.465331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:06.486497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f59e2-188b-45f0-bc0f-277c28228752 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:06.519194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:06.519889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.520174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.520448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.521185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:06.521493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:06.522851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:06.531140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 579/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:49:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:06.536937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:06.627255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:06.633496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e0832-7e01-4f21-b602-0ff39425708a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:06.666154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:06.667004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.667287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.667591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] PATH_INFO: `/v3/users/5a559db65c3d41ecb78418c117dc9185` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.668761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] RBAC: Authorizing `identity:get_user(user_id=5a559db65c3d41ecb78418c117dc9185)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:06.670174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:06.672059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 572/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:49:06 2022] GET /identity/v3/users/5a559db65c3d41ecb78418c117dc9185 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:06.677711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:06.775181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce741fb-1faa-4130-9b34-f75394d70766 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:06.807828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a06199-0db7-4d0a-a600-8dc9af357e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:06.808686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06199-0db7-4d0a-a600-8dc9af357e46 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.808969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06199-0db7-4d0a-a600-8dc9af357e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.809244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06199-0db7-4d0a-a600-8dc9af357e46 None admin] PATH_INFO: `/v3/projects/00bbb8057bbf48af917aff638bc581f8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.810405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a06199-0db7-4d0a-a600-8dc9af357e46 None admin] RBAC: Authorizing `identity:get_project(project_id=00bbb8057bbf48af917aff638bc581f8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:06.811857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a06199-0db7-4d0a-a600-8dc9af357e46 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:06.813530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 580/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:49:06 2022] GET /identity/v3/projects/00bbb8057bbf48af917aff638bc581f8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:06.819377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:06.918817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2b5a1-58b1-4f73-b57f-e6fd57984559 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:06.952389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed700be-2ac4-471b-a122-2251d986528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:06.953260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed700be-2ac4-471b-a122-2251d986528f None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:06.953540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed700be-2ac4-471b-a122-2251d986528f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:06.953912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed700be-2ac4-471b-a122-2251d986528f None admin] PATH_INFO: `/v3/projects/00bbb8057bbf48af917aff638bc581f8/users/5a559db65c3d41ecb78418c117dc9185/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:06.956241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed700be-2ac4-471b-a122-2251d986528f None admin] RBAC: Authorizing `identity:create_grant(project_id=00bbb8057bbf48af917aff638bc581f8, user_id=5a559db65c3d41ecb78418c117dc9185, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:06.957648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed700be-2ac4-471b-a122-2251d986528f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:07.046740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 573/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:49:06 2022] PUT /identity/v3/projects/00bbb8057bbf48af917aff638bc581f8/users/5a559db65c3d41ecb78418c117dc9185/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:49:09.151676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53c702-a4a2-499c-9ce4-c47e0528298b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:09.152237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53c702-a4a2-499c-9ce4-c47e0528298b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:09.152237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53c702-a4a2-499c-9ce4-c47e0528298b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:09.153611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 581/1154] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:09.158656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:09.159095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:09.159201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:09.176509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:09.177674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:09.194257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:09.244270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 574/1155] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:49:09.267337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53c702-a4a2-499c-9ce4-c47e0528298b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:09.278555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:09.368728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee53c702-a4a2-499c-9ce4-c47e0528298b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:09.401863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:09.402589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:09.402874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:09.403151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:09.413403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:49:09.421699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:09.422789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:49:09.423731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 582/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:09.428901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:09.526784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6154ad-7964-42c9-b8aa-ebedba214732 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:09.562239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:09.562989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:09.563279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:09.563588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:09.564334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:09.564647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:09.566014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:09.574328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 575/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:49:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:09.580491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:09.646194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:09.680146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5d0a4-a492-408b-b6ae-893ae1493c25 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:09.712973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:09.713862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:09.714146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:09.714420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] PATH_INFO: `/v3/users/5a559db65c3d41ecb78418c117dc9185` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:09.715625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] RBAC: Authorizing `identity:get_user(user_id=5a559db65c3d41ecb78418c117dc9185)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:09.717047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:09.718676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 583/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:49:09 2022] GET /identity/v3/users/5a559db65c3d41ecb78418c117dc9185 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:09.724430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:09.741734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:09.822118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a2909-1bed-4e22-9162-5d13cbdf2911 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:09.854739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:09.855393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:09.855683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:09.855956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:09.920078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:09.921751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:09.936024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:10.020511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 576/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:49:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:49:10.027163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:10.124599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326e2f0-8f28-4fd1-bdda-1473ea7a6795 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:10.158639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb1e04-41c3-4d65-a96e-58d546446b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:10.159519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1e04-41c3-4d65-a96e-58d546446b1b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:10.159806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1e04-41c3-4d65-a96e-58d546446b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:10.160084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb1e04-41c3-4d65-a96e-58d546446b1b None admin] PATH_INFO: `/v3/projects/00bbb8057bbf48af917aff638bc581f8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:10.161260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1e04-41c3-4d65-a96e-58d546446b1b None admin] RBAC: Authorizing `identity:get_project(project_id=00bbb8057bbf48af917aff638bc581f8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:10.162688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb1e04-41c3-4d65-a96e-58d546446b1b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:10.164598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 584/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:49:10 2022] GET /identity/v3/projects/00bbb8057bbf48af917aff638bc581f8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:10.170920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:10.270737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b3e15-d420-4e05-83ea-b68d9a0e2321 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:10.285200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:10.305763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:10.306405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:10.306686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:10.306961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:10.307805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=00bbb8057bbf48af917aff638bc581f8, user.id=5a559db65c3d41ecb78418c117dc9185) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:10.308114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:10.309513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c7b58-f199-4880-aac6-b1914c3efff6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:10.315403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 577/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:49:10 2022] GET /identity/v3/role_assignments?user.id=5a559db65c3d41ecb78418c117dc9185&scope.project.id=00bbb8057bbf48af917aff638bc581f8&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:12.521048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01848e4f-bc55-4c03-be32-6355bac3f1b3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:12.521552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01848e4f-bc55-4c03-be32-6355bac3f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:12.521552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01848e4f-bc55-4c03-be32-6355bac3f1b3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:12.523627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 585/1162] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:12.529608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:12.530022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:12.530169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:12.547355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:12.548557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:12.613633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 578/1163] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:12.635736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01848e4f-bc55-4c03-be32-6355bac3f1b3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:12.639514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:12.734031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01848e4f-bc55-4c03-be32-6355bac3f1b3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:12.765953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:12.766590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:12.766874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:12.767148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:12.834184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:12.836525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:12.839144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:12.941400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 586/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:49:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:49:12.948700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:13.048849 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec24da7-9143-48c3-9b2d-ae81b89e4ba9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:13.058130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:13.082452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:13.083251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:13.083670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:13.083952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:13.091512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:13.092895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d3c69d62-ea89-4d60-8277-ff29b44a2b83 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:13.106771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:13.108017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:49:13.108898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 579/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:49:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:13.113922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:13.210241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81e4f9-3561-4bdc-9818-30f660354b8d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:13.213092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:13.243634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:13.244301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:13.244590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:13.244869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:13.245691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:13.246292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:13.248022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8cf1-404d-4bfd-9cf7-20cb9a6f0bee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:13.255684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 587/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Thu Jun 16 17:49:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:15.352237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065e27d-6a7b-45e1-becb-258d031fc10b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:15.352780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065e27d-6a7b-45e1-becb-258d031fc10b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:15.352780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065e27d-6a7b-45e1-becb-258d031fc10b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:15.354176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 580/1167] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:15.359554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285859ec-118c-40c0-ac59-6e842cd345de None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:15.359973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285859ec-118c-40c0-ac59-6e842cd345de None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:15.360115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285859ec-118c-40c0-ac59-6e842cd345de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:15.376779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-285859ec-118c-40c0-ac59-6e842cd345de None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:15.377983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-285859ec-118c-40c0-ac59-6e842cd345de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:15.452390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 588/1168] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:15.475823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065e27d-6a7b-45e1-becb-258d031fc10b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:15.578331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065e27d-6a7b-45e1-becb-258d031fc10b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:15.611495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad1f11-b12a-4c7c-a559-cc0c0105df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:15.612152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1f11-b12a-4c7c-a559-cc0c0105df13 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:15.612438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1f11-b12a-4c7c-a559-cc0c0105df13 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:15.612717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1f11-b12a-4c7c-a559-cc0c0105df13 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:15.613458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad1f11-b12a-4c7c-a559-cc0c0105df13 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:15.614812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad1f11-b12a-4c7c-a559-cc0c0105df13 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:15.698995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 581/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Thu Jun 16 17:49:15 2022] POST /identity/v3/domains => generated 267 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:49:17.873231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fba5-c35b-4d7d-bba2-e75ae3b33dba None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:17.873759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fba5-c35b-4d7d-bba2-e75ae3b33dba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:17.873866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949fba5-c35b-4d7d-bba2-e75ae3b33dba None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:17.875160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 589/1170] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:17.880567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:17.880942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:17.881125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:17.897774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:17.899082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:17.922990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:17.965840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 582/1171] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:17.989641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fba5-c35b-4d7d-bba2-e75ae3b33dba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:18.084029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:18.089149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949fba5-c35b-4d7d-bba2-e75ae3b33dba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:18.124257 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cba37-6eb4-40f0-ba61-7eae68d831aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:18.125019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cba37-6eb4-40f0-ba61-7eae68d831aa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:18.125297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cba37-6eb4-40f0-ba61-7eae68d831aa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:18.125571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cba37-6eb4-40f0-ba61-7eae68d831aa None admin] PATH_INFO: `/v3/domains/36715e5246714593911c948cbdcd672e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:18.139337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cba37-6eb4-40f0-ba61-7eae68d831aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=36715e5246714593911c948cbdcd672e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:18.140798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cba37-6eb4-40f0-ba61-7eae68d831aa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:18.142492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 590/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Jun 16 17:49:17 2022] GET /identity/v3/domains/36715e5246714593911c948cbdcd672e => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:18.150142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:18.248713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8093f-0046-4cd0-9aa1-13ac363131a9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:18.282973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f152c1-1d86-4bde-b0c7-9fa1fde69518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:18.283650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f152c1-1d86-4bde-b0c7-9fa1fde69518 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:18.283934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f152c1-1d86-4bde-b0c7-9fa1fde69518 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:18.284206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f152c1-1d86-4bde-b0c7-9fa1fde69518 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:18.284904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f152c1-1d86-4bde-b0c7-9fa1fde69518 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:18.286319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f152c1-1d86-4bde-b0c7-9fa1fde69518 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:18.402644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 583/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:49:18 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:49:20.564336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b278b7-b076-49a7-a9f1-dd383ea9cc1a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:20.564840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b278b7-b076-49a7-a9f1-dd383ea9cc1a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:20.564840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b278b7-b076-49a7-a9f1-dd383ea9cc1a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:20.566172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 591/1174] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:20.572392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:20.572798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:20.572941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:20.589057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:20.590229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:20.592926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:20.657105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 584/1175] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:20.682212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b278b7-b076-49a7-a9f1-dd383ea9cc1a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:20.788699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b278b7-b076-49a7-a9f1-dd383ea9cc1a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:20.822745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322b7cf-39d1-440c-8fe8-5ca2ca183704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:20.823561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b7cf-39d1-440c-8fe8-5ca2ca183704 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:20.823841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b7cf-39d1-440c-8fe8-5ca2ca183704 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:20.824114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322b7cf-39d1-440c-8fe8-5ca2ca183704 None admin] PATH_INFO: `/v3/domains/36715e5246714593911c948cbdcd672e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:20.825261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322b7cf-39d1-440c-8fe8-5ca2ca183704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=36715e5246714593911c948cbdcd672e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:20.826669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322b7cf-39d1-440c-8fe8-5ca2ca183704 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:20.828310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 592/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Thu Jun 16 17:49:20 2022] GET /identity/v3/domains/36715e5246714593911c948cbdcd672e => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:20.834095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:20.930347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220acfee-5c14-45ba-aa3c-f0cc265e2b8d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:20.962591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b066e-5492-41a7-bab5-02495e18f14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:20.963240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b066e-5492-41a7-bab5-02495e18f14f None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:20.963676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b066e-5492-41a7-bab5-02495e18f14f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:20.963957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b066e-5492-41a7-bab5-02495e18f14f None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:20.964663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b066e-5492-41a7-bab5-02495e18f14f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:20.966072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b066e-5492-41a7-bab5-02495e18f14f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:20.975716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:21.124547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 585/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:49:20 2022] POST /identity/v3/users => generated 311 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:49:23.315897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c4cd-ef60-4c20-a5a8-b25f5ac995a8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:23.316467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c4cd-ef60-4c20-a5a8-b25f5ac995a8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:23.316467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877c4cd-ef60-4c20-a5a8-b25f5ac995a8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:23.317797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 593/1178] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:23.323625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:23.324048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:23.324164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:23.340761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:23.341927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:23.406321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 586/1179] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:23.431122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877c4cd-ef60-4c20-a5a8-b25f5ac995a8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:23.458137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:23.533060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877c4cd-ef60-4c20-a5a8-b25f5ac995a8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:23.566468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:23.567247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:23.567555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:23.567830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:23.578356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:49:23.586390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:23.587536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:49:23.588584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 594/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:23.593727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:23.690038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26ebc-290e-4783-a75a-2b756c207978 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:23.719386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:23.728670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:23.729338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:23.732740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:23.732740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:23.732740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:23.732740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:23.732740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:23.740147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 587/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:49:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:23.746169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:23.828632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:23.843242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5941-809b-4d3a-931e-ebe39d27fda6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:23.877317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:23.878174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:23.878474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:23.878770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] PATH_INFO: `/v3/users/9817853582214f608497f2da6230d531` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:23.894377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] RBAC: Authorizing `identity:get_user(user_id=9817853582214f608497f2da6230d531)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:23.905068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:23.906775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 595/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:49:23 2022] GET /identity/v3/users/9817853582214f608497f2da6230d531 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:23.912614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:24.010304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f68518-c2dc-494e-a7b4-03303495a555 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:24.043925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:24.044558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:24.045098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:24.045374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:24.099138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:24.110916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:24.112379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:24.219166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 588/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:49:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:49:24.225242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:24.331733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921e311-4b31-4427-8a56-206297c9f845 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:24.364833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8de32-4910-433f-b011-b43ec526a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:24.365621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8de32-4910-433f-b011-b43ec526a002 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:24.365909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8de32-4910-433f-b011-b43ec526a002 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:24.366184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8de32-4910-433f-b011-b43ec526a002 None admin] PATH_INFO: `/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:24.367443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8de32-4910-433f-b011-b43ec526a002 None admin] RBAC: Authorizing `identity:get_project(project_id=9e619589e8224fe9afbd7dbe1de0f9fb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:24.368871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8de32-4910-433f-b011-b43ec526a002 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:24.370562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 596/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:49:24 2022] GET /identity/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:24.376828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:24.462476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:24.474019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8988b4-78ba-4956-8f13-73f2bfb0a670 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:24.507717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:24.508362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:24.508642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:24.508914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:24.509729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=9e619589e8224fe9afbd7dbe1de0f9fb, user.id=9817853582214f608497f2da6230d531) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:24.510039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:24.511473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521bee3-d163-442d-8cac-7e1f1bed7104 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:24.516821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 589/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:49:24 2022] GET /identity/v3/role_assignments?user.id=9817853582214f608497f2da6230d531&scope.project.id=9e619589e8224fe9afbd7dbe1de0f9fb&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:26.616540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47880b9-89fc-4bff-954d-5a6dd7c1102d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:26.617053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47880b9-89fc-4bff-954d-5a6dd7c1102d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:26.617053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47880b9-89fc-4bff-954d-5a6dd7c1102d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:26.618377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 597/1186] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:26.624156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005413-2896-4fba-859a-38053664b6dc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:26.624588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005413-2896-4fba-859a-38053664b6dc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:26.624972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005413-2896-4fba-859a-38053664b6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:26.646574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-52005413-2896-4fba-859a-38053664b6dc None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:26.648417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-52005413-2896-4fba-859a-38053664b6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:26.716528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 590/1187] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:26.739511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47880b9-89fc-4bff-954d-5a6dd7c1102d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:26.754347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:26.841691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47880b9-89fc-4bff-954d-5a6dd7c1102d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:26.876159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:26.876861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:26.877139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:26.877406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:26.887674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:49:26.895005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:26.896101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:49:26.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 598/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:26.902874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52005413-2896-4fba-859a-38053664b6dc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:26.991013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:27.008189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52005413-2896-4fba-859a-38053664b6dc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:27.042815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:27.043504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:27.043789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:27.044086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:27.045072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:27.045583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:27.047204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:27.056030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 591/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:27.062157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:27.130450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:27.159475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13afdef-9127-47f5-9c04-a275ee76253e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:27.191826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:27.192685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:27.192968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:27.193245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] PATH_INFO: `/v3/users/9817853582214f608497f2da6230d531` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:27.194926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] RBAC: Authorizing `identity:get_user(user_id=9817853582214f608497f2da6230d531)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:27.196462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:27.198108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 599/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:49:27 2022] GET /identity/v3/users/9817853582214f608497f2da6230d531 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:27.203887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:27.302009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68760071-8cf3-435f-9e82-3cb6415b00dd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:27.335951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405fa1b-2a37-460b-a70f-a407405bff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:27.336830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405fa1b-2a37-460b-a70f-a407405bff9c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:27.337118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405fa1b-2a37-460b-a70f-a407405bff9c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:27.337398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405fa1b-2a37-460b-a70f-a407405bff9c None admin] PATH_INFO: `/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:27.339145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405fa1b-2a37-460b-a70f-a407405bff9c None admin] RBAC: Authorizing `identity:get_project(project_id=9e619589e8224fe9afbd7dbe1de0f9fb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:27.340622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405fa1b-2a37-460b-a70f-a407405bff9c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:27.342360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 592/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:49:27 2022] GET /identity/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:27.353228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:27.457971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56164250-ba4c-4e2a-8f3e-0ec794ea1e9a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:27.490512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd08cae-1f54-4945-bf67-aaef3125567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:27.492078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd08cae-1f54-4945-bf67-aaef3125567e None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:27.492358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd08cae-1f54-4945-bf67-aaef3125567e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:27.492634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd08cae-1f54-4945-bf67-aaef3125567e None admin] PATH_INFO: `/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb/users/9817853582214f608497f2da6230d531/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:27.494575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd08cae-1f54-4945-bf67-aaef3125567e None admin] RBAC: Authorizing `identity:create_grant(project_id=9e619589e8224fe9afbd7dbe1de0f9fb, user_id=9817853582214f608497f2da6230d531, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:27.496037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd08cae-1f54-4945-bf67-aaef3125567e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:27.566812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:27.572417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 600/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:49:27 2022] PUT /identity/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb/users/9817853582214f608497f2da6230d531/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:49:29.683462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f898-3981-41fb-8e14-3bf08551feb2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:29.684056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f898-3981-41fb-8e14-3bf08551feb2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:29.684802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94f898-3981-41fb-8e14-3bf08551feb2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:29.686193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 593/1193] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:49:29.691650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:29.692119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:29.692231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:29.711661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:49:29.715037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:49:29.792361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 601/1194] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:49:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:49:29.822254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f898-3981-41fb-8e14-3bf08551feb2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:29.839234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:29.937319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94f898-3981-41fb-8e14-3bf08551feb2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:29.975305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:29.976705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:29.976990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:29.977273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:29.988196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:49:29.998080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:49:30.017882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:49:30.018910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 594/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:49:30.019721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:30.150056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71e591-48cd-426f-860d-7d8b0735fd10 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:30.189103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:30.199026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:30.204583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:30.204583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:30.204583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:30.204583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:30.204583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:30.204583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:30.215107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 602/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:49:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:30.221107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:30.303669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:30.336851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e7821-5507-4fb2-83f0-27fa897b1868 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:30.375113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:30.376009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:30.376295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:30.376571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] PATH_INFO: `/v3/users/9817853582214f608497f2da6230d531` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:30.377762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] RBAC: Authorizing `identity:get_user(user_id=9817853582214f608497f2da6230d531)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:30.379235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:30.380906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 595/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Thu Jun 16 17:49:30 2022] GET /identity/v3/users/9817853582214f608497f2da6230d531 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:30.388639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:30.493682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c3104-c884-4f34-b709-0eafe32a3adf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:30.525950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:30.526593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:30.526879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:30.527155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:30.582732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:30.592427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:30.594420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:30.690794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 603/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:49:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:49:30.697630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:30.795664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d50d9-57a1-4215-9e49-5c851c533d8f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:30.828857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08587aec-693b-40d9-ac76-bb009c226251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:30.829699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08587aec-693b-40d9-ac76-bb009c226251 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:30.829981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08587aec-693b-40d9-ac76-bb009c226251 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:30.830256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08587aec-693b-40d9-ac76-bb009c226251 None admin] PATH_INFO: `/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:30.831459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08587aec-693b-40d9-ac76-bb009c226251 None admin] RBAC: Authorizing `identity:get_project(project_id=9e619589e8224fe9afbd7dbe1de0f9fb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:30.832872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08587aec-693b-40d9-ac76-bb009c226251 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:30.835089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 596/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Thu Jun 16 17:49:30 2022] GET /identity/v3/projects/9e619589e8224fe9afbd7dbe1de0f9fb => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:30.841430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:49:30.931939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:30.940235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6955a60-ffa1-4ee2-940e-8416bce66e9d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:49:30.973268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:49:30.973918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:49:30.974203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:49:30.974481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:49:30.975508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=9e619589e8224fe9afbd7dbe1de0f9fb, user.id=9817853582214f608497f2da6230d531) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:49:30.976067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:49:30.977794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4700731-8b8e-4b3f-9b43-aa2637910fb8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:49:30.983888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 604/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:49:30 2022] GET /identity/v3/role_assignments?user.id=9817853582214f608497f2da6230d531&scope.project.id=9e619589e8224fe9afbd7dbe1de0f9fb&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:49:40.824637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:49:40.981252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=22 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:46.731131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa07ff7-fa51-4c69-85b8-0fae13247cb2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:46.731771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa07ff7-fa51-4c69-85b8-0fae13247cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:46.731771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa07ff7-fa51-4c69-85b8-0fae13247cb2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:46.733082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 597/1201] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:50:46.738717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:46.739154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:46.739264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:46.756774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:46.757970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:46.830545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 605/1202] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:50:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:50:46.855561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa07ff7-fa51-4c69-85b8-0fae13247cb2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:46.957877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa07ff7-fa51-4c69-85b8-0fae13247cb2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:46.991084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:46.991850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:46.992137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:46.992422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:47.002752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:47.004215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:47.021788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:47.022917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:47.024318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:47.024318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:47.024318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:47.024318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:50:47.024318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 598/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:47.028988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:47.127914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05e46-ddda-4f89-b23c-d2534d813ed8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:47.157212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:47.162900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:47.163592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:47.163881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:47.164427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:47.165143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:47.165451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:47.166869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0eb0e-6cb6-4447-b021-0140275dd468 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:47.178282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 606/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:47.184238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:47.238697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:47.285814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a869b8-3341-4e55-a46c-5847ec9e1ebe None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:47.318959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311863da-0fd3-4233-a045-a68fce61ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:47.319652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311863da-0fd3-4233-a045-a68fce61ce2f None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:47.319940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311863da-0fd3-4233-a045-a68fce61ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:47.320217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311863da-0fd3-4233-a045-a68fce61ce2f None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:47.320928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311863da-0fd3-4233-a045-a68fce61ce2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:47.322399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311863da-0fd3-4233-a045-a68fce61ce2f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:47.420287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 599/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Thu Jun 16 17:50:47 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:50:49.614879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594596b-8f94-4e7a-b34c-cbe4aab672c3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.615459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594596b-8f94-4e7a-b34c-cbe4aab672c3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.615459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594596b-8f94-4e7a-b34c-cbe4aab672c3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.616769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 607/1206] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:50:49.622190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.622619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.622752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.641447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:49.642669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:49.717698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 600/1207] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:50:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:50:49.742254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594596b-8f94-4e7a-b34c-cbe4aab672c3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.844132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594596b-8f94-4e7a-b34c-cbe4aab672c3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:49.877927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:49.878659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:49.878947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:49.879228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:49.889644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:50:49.898575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:49.899737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:49.900876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 608/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:50:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:49.906041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:49.945106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.004284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133d962-3cd7-4757-95aa-b7e2c4d4b2a2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.037811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.040307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.040307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.040307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.040307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:50.040307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.041696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.049786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 601/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:50:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:50.055651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.105308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.154526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa683cb-42ae-453e-b95f-5bf545b60829 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.190487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.191551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.192023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.192810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.202581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.204176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:50.221414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:50.222500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:50.223884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:50.223884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:50.223884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:50.223884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:50.223884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 609/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:50.231758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.328521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.331410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de8c8c-4fb7-4c08-87e0-95a99ba02d0f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.365158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.365815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.366095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.366368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.367073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:50.367409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.368758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.381500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 602/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:50.386870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.487388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe3ace-be48-42f1-b70b-f85c0a8694d9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.488926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.521547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.522348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.522633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.522912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.530149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.531888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:50:50.546599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:50.547768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:50.548970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 610/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:50.553613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.652175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258abf8f-3f87-443f-be68-635ea459c835 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.685655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.686316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.686601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.687456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.688801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:50.689350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.691445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:50.711337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 603/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jun 16 17:50:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:50.717267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:50.818474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606d0f-3b46-479c-8bd2-4dc0e020abc7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:50.852521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:50.853142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:50.853441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:50.853708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:50.883752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:50.920977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:50.923138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.019355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 611/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:50:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:50:51.025185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.044709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.126205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63641-67c0-41a8-b64d-2a96c7fe0050 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.163307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.164394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.164680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.164954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.177382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.179734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:51.197621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:51.203958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:51.205122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:51.205122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:51.205122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:51.205122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:50:51.205122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 604/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:51.205497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.265445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.303824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123669d-4d06-460c-9ba4-ac2f7f679d79 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.336380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.337040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.337325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.337600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.338310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:51.338616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.340045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.351940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 612/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:51.357366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.440706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.474663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9efb82-a82a-4e32-83ec-c1b2a2f2820a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.507723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.508517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.508801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.509080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.517060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.518490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:50:51.533263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:51.534365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:50:51.535602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 605/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:51.540627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.638652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d4a80-6702-4cae-8acf-a6aeb149f52a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.661144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.672576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.673241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.673527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.673803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.674589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:51.674943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.676354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127b968-0faa-44ae-94b0-80faafb85310 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.691138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 613/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:51.694366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:51.797002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15349d-0d56-4cb7-b2de-667e04150d87 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:51.828932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:51.834685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:51.835342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:51.835710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:51.835990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:51.836821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=efaf16b0733b426da91c5f5ae08deaac) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:51.837134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:51.838556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455799a-7566-4a11-8a4a-4bd602f1af02 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:51.844247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 606/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:50:51 2022] GET /identity/v3/role_assignments?user.id=efaf16b0733b426da91c5f5ae08deaac&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:53.965760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5897d97-020f-4421-be0b-ac027a589900 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.966312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5897d97-020f-4421-be0b-ac027a589900 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.966312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5897d97-020f-4421-be0b-ac027a589900 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.968105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 614/1220] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:50:53.973400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:53.973783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:53.973963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:53.991094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:53.992325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:54.058469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 607/1221] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:50:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:50:54.081691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5897d97-020f-4421-be0b-ac027a589900 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.187341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5897d97-020f-4421-be0b-ac027a589900 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.221395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.222136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.222409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.222687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.232510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:50:54.240856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:54.241954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:54.243020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 615/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:50:54.248422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.346348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9d894-c8d8-4361-88ba-3cd70525a716 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.369631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.380096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.380800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.381047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.381323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.382067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:54.382379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.383777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.392657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 608/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:54.398704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.443525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.500097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e37ab-7863-4a40-89c2-57d16471a204 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.533475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.534260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.534547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.534828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.545423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.547565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:54.564160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:54.575942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:54.577593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:54.577593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:54.577593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:54.577593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:54.577593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 616/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:54.577890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.671622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc4b1f-4067-4ded-95d8-aa63eb3dbe9b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.704479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.705138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.705417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.705687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.706394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:54.706695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.708086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.720912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 609/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:54.726598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.826219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.833254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42ab50-a5ec-433e-be6b-4c0b5282f2c7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:54.865077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:54.868281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:54.868281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:54.868281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:54.874687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:54.876248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:50:54.890700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:54.891820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:54.892860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 617/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:54.897828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:54.944357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:54.995234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ad916-7320-419f-9006-ff0e517e57a4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.030335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.031166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.031472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.031750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.032572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:55.032880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.034287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.051300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 610/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jun 16 17:50:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:55.057000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.157377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8444-7c5c-4e14-96b6-fb525d463bd8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.192590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.193380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.193665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.193950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.204307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.205733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:55.222448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:55.223568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:55.224847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:55.224847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:55.224847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:55.224847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:55.224847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 618/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:55.229426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.339386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73770511-b48e-481d-a57b-d7718b277d78 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.341835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:55.372628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.373288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.381798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.381798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.381798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:55.381798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.381798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.387773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 611/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:55.394139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.403483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:55.493172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f03650-d0f7-4812-bc0a-dd58c78b15aa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.528638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.529539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.530617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.531152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.538494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.539959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:50:55.554588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:55.555699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:55.561456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 619/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:50:55.562543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.661025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e258db-0d1f-4256-9d4c-f50c714cfb8f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.693924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.694585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.694870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.695147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.695968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:55.696279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.697708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5c7b2-659c-4224-b5bf-4301a3394689 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:55.709405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 612/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:55.715651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:55.781679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:55.817174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d03ee-d8fd-4d58-9c14-8bb8ee616b0c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:55.850932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddd44c-13ea-400d-b5ac-5cbab79fdeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:55.851839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd44c-13ea-400d-b5ac-5cbab79fdeba None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:55.852123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd44c-13ea-400d-b5ac-5cbab79fdeba None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:55.852403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddd44c-13ea-400d-b5ac-5cbab79fdeba None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/efaf16b0733b426da91c5f5ae08deaac/roles/777a339245484c7fbe89538db336a995` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:55.867523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddd44c-13ea-400d-b5ac-5cbab79fdeba None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=efaf16b0733b426da91c5f5ae08deaac, role_id=777a339245484c7fbe89538db336a995)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:55.869070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddd44c-13ea-400d-b5ac-5cbab79fdeba None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:56.065826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 620/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:50:55 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/efaf16b0733b426da91c5f5ae08deaac/roles/777a339245484c7fbe89538db336a995 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:50:58.174055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16597985-3851-4ccd-9a35-d08c7995e204 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:58.174592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16597985-3851-4ccd-9a35-d08c7995e204 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:58.174592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16597985-3851-4ccd-9a35-d08c7995e204 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:58.176005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 613/1233] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:50:58.181240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:58.181622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:58.186980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:58.197964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:50:58.199152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:50:58.268288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 621/1234] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:50:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:50:58.293281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16597985-3851-4ccd-9a35-d08c7995e204 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:58.318468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:58.392976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16597985-3851-4ccd-9a35-d08c7995e204 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:58.427005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:58.427769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:58.428114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:58.428393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:58.438270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:50:58.446134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:58.451330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:50:58.452303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 614/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Thu Jun 16 17:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:58.454117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:58.500120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:58.552921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758d425-e4e0-46a8-b3c0-c028f204dac9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:58.586683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:58.587349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:58.587687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:58.587966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:58.588707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:58.589015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:58.590459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:58.598904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 622/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Thu Jun 16 17:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:58.604726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:58.691568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:58.707572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992c428-5e5c-46b8-abe4-834591e801a6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:58.739922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:58.742387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:58.742387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:58.742387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:58.752307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:58.753740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:58.770979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:58.772138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:58.773240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:58.773240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:58.773240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:58.773240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:50:58.773240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 615/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:58.778395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:58.879008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df02b4-8d52-4095-aa9a-a00266f1434c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:58.911585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:58.912249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:58.912535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:58.912813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:58.913788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:58.914098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:58.915496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:58.926984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 623/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:58.933778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:58.933931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:59.032284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc207f-78db-422b-be6b-3c9928474a84 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:59.066190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:59.066982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:59.067263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:59.067564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:59.070303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:59.075263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:59.076705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:50:59.090799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:59.092011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:50:59.093096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 616/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:59.098042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:59.197425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a827e-84f5-4be2-846f-6edaf552e9a0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:59.231748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:59.232423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:59.232709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:59.232988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:59.234456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:59.234762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:59.236214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:59.253343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 624/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jun 16 17:50:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:59.259167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:59.357538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab392-a88a-4047-a4e0-055f392646da None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:59.391109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:59.391778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:59.392059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:59.392331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:59.452714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:59.458987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:59.460443 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:59.558972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 617/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:50:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:50:59.565072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:59.594237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:59.666407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d000ed-68e7-4af1-a8f8-003930aff1ff None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:59.699596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:59.700395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:59.700676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:59.700952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:59.711624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:59.713065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:50:59.730640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:50:59.731882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:50:59.733188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:50:59.733188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:50:59.733188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:50:59.733188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:50:59.733188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 625/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:50:59.737630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:59.833049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:59.838765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8836ab-7864-4092-bad2-7d4aa46153e4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:50:59.875736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:50:59.876406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:50:59.876693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:50:59.876972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:50:59.877765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:50:59.878116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:50:59.879986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:50:59.892459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 618/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:50:59.898191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:50:59.975595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:50:59.995915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d49e33-0dd5-41d5-8a66-c1ecf064fa9b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:00.029915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:00.030808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:00.031094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:00.031400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:00.037788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:00.039498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:51:00.052954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:00.054008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:00.055063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 626/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:00.060052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:00.170039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443cd82-176c-42ee-9de8-e2a6494f7df9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:00.209044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:00.211093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:00.211093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:00.211093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:00.211414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:00.211966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:00.214341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef12df-ce68-45a3-a451-ed71c079f110 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:00.230275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 619/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:51:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:00.236871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:00.339419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f2780-03bc-4131-b223-0f0ee79bae6c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:00.372984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:00.373642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:00.373933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:00.374239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:00.375619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] RBAC: Adding query filter params (role.id=777a339245484c7fbe89538db336a995, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=efaf16b0733b426da91c5f5ae08deaac) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:00.375932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:00.377429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2384a1-ce98-45a5-9b8a-e3f8bcd447c9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:00.383272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 627/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:51:00 2022] GET /identity/v3/role_assignments?user.id=efaf16b0733b426da91c5f5ae08deaac&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=777a339245484c7fbe89538db336a995 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:02.596480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25542f8a-62ba-4ae3-a6b9-c30b8652a732 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:02.597002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25542f8a-62ba-4ae3-a6b9-c30b8652a732 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:02.597002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25542f8a-62ba-4ae3-a6b9-c30b8652a732 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:02.598337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 620/1247] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:02.603757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:02.608812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:02.608812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:02.609725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:02.622291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:02.623528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:02.690995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 628/1248] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:51:02.715314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25542f8a-62ba-4ae3-a6b9-c30b8652a732 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:02.733827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:02.816538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25542f8a-62ba-4ae3-a6b9-c30b8652a732 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:02.851404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:02.852170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:02.852453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:02.852731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:02.862766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:51:02.870672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:02.871863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:02.873089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 621/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:51:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:02.878134 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:02.976185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5defcd-a812-43bc-b926-aa6db58912dd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:03.006094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:03.010871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:03.011630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:03.011915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:03.012194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:03.012933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:03.013242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:03.015244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:03.023331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 629/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:51:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:03.029280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:03.106610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:03.129441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1b71-6739-4160-9aa2-6905b0f76a29 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:03.161995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:03.163031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:03.163315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:03.163621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:03.174140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:03.175606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:51:03.192669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:03.193790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:03.195031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:51:03.195031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:51:03.195031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:03.195031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:03.195031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 622/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:03.199751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:03.299512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caccc17-8d89-4593-a6a2-08f30946679a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:03.338124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:03.338797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:03.339084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:03.339377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:03.340112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:03.340426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:03.341858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:03.353580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 630/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:03.360025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:03.459880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ccbce-af48-4dca-9d24-e7c19e921b44 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:03.485753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:03.493329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:03.494120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:03.494407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:03.494685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:03.502593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:03.504066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:51:03.518303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:03.519446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:03.520726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 623/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:51:03.525527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:03.580749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:03.624836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b527b-c028-4cfc-a2d8-d7045a5173f5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:03.657128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:03.658138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:03.658424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:03.658701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:03.659564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:03.659878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:03.661300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:03.678965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 631/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jun 16 17:51:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:03.684662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:03.785388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c25a5-da5c-4a51-ac51-ad7e2e2b8568 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:03.818227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:03.818876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:03.819165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:03.819477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:03.874722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:03.886581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:03.888000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:03.984710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 624/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:51:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:51:03.990760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:04.089204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b5c53-26de-4ff3-9c4b-b87fc162c3cc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:04.102566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:04.122412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:04.125130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:04.125130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:04.125130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:04.133478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:04.134904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:51:04.153129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:04.154652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:04.156642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:51:04.156642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:51:04.156642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:04.156642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:04.156642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 632/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:04.160567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:04.246117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:04.258088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c31a03-4052-4aa4-abde-d5cb676c926c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:04.292750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:04.294777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:04.294777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:04.294777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:04.294777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:04.295194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:04.297247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:04.308066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 625/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:04.314509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:04.412757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9806c8-1681-4b7c-b35b-27082dd0148d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:04.446711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:04.447522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:04.447804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:04.448076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:04.454806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:04.456245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:51:04.470761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:04.473293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:04.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 633/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:04.478032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:04.578416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b05f03-a736-4ec4-bc25-991a4fe3064e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:04.611791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:04.612447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:04.612735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:04.613012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:04.613798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:04.614110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:04.615551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc0a8-10e3-4a21-a9c5-f32d69a456b2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:04.627729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 626/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:04.633963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:04.661826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:04.735844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb536a8-0774-4665-8541-fa7cca55149c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:04.767756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:04.768410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:04.768707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:04.768976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:04.769810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=efaf16b0733b426da91c5f5ae08deaac) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:04.770125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:04.771606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352827-98a7-4ade-9453-bd1d65df7ddc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:04.776866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 634/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:51:04 2022] GET /identity/v3/role_assignments?user.id=efaf16b0733b426da91c5f5ae08deaac&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:51:06.907882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ef232-2b6a-4d11-bb5f-bc8607d515db None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.908441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ef232-2b6a-4d11-bb5f-bc8607d515db None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.908441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ef232-2b6a-4d11-bb5f-bc8607d515db None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.909809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 627/1261] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:06.915407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:06.915833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:06.915975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:06.932868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:06.934719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:07.004229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 635/1262] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:51:07.027589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ef232-2b6a-4d11-bb5f-bc8607d515db None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.039405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.128994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ef232-2b6a-4d11-bb5f-bc8607d515db None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.161582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.162553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.162841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.163118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.173679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:51:07.181322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:07.182503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:07.183542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 628/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:07.189529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.208319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.288157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530dbfe-cb8a-4d2a-a1af-aa2fbe88489a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.323191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.323883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.324169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.324442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.325172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:07.325479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.327025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.334864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 636/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:07.340654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.411194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.444144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb08753-ae99-4557-b463-4edae0c9c3ac None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.476182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.476967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.477251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.477529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.488304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.489725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:51:07.506222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:07.511159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:07.512293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:51:07.512293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:51:07.512293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:07.512293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:07.512293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 629/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:51:07.513354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.595594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.614708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae3840-2d23-4655-a27f-3a6f7a9efd04 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.647380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.648098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.648388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.648678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.649391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:07.649702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.651184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.663596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 637/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:07.669297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.770007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bcf0d-1aeb-4a98-85d6-a881e14cd2b5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.790220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.804399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.805188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.805468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.805739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.813132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.814543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:51:07.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:07.830268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:07.831605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 630/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:07.837434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:07.934546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244a339-61fa-4a31-a29e-d3b751d5e454 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:07.968457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:07.969131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:07.969447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:07.969723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:07.970559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:07.970866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:07.972305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:07.981118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:07.989417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 638/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:07.995051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.095619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051d757-b2c2-42d4-8c8f-d830bbe3f012 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.127746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.128518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.128797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.129065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.140177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.141617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:51:08.158898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:08.160138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:08.161252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:51:08.161252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:51:08.161252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:08.161252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:08.161252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 631/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:08.166182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.266675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138049e1-9f0d-4caf-af7a-ad70639bd989 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.300597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.301264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.301551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.301834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.302560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:08.302892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.304357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.316185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 639/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:08.321917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.353281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:08.421545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3f322-21d6-4ea3-8929-f58fe5bc8bb8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.455044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.455847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.456446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.456716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.463756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.465227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:51:08.479470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:08.480567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:08.481792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 632/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:08.486637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.545235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:08.587117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b78ce8-874a-451d-9e76-509ebb06392b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.620744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.621338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.621624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.621900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.622677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:08.622987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.625166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3554d-8b11-4b06-8cad-a403f8f8251e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.636891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 640/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:08.643000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:08.733876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:08.742347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c8333-c216-4af8-81d9-af7d9236a042 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:08.775528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1466082-72c7-4268-9d71-5c93eec14a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:08.776402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1466082-72c7-4268-9d71-5c93eec14a17 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:08.776691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1466082-72c7-4268-9d71-5c93eec14a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:08.776968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1466082-72c7-4268-9d71-5c93eec14a17 None admin] PATH_INFO: `/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/efaf16b0733b426da91c5f5ae08deaac/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:08.778900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1466082-72c7-4268-9d71-5c93eec14a17 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca560e4628b041ee8e8132b2f35b4c4c, user_id=efaf16b0733b426da91c5f5ae08deaac, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:08.780547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1466082-72c7-4268-9d71-5c93eec14a17 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:08.916002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 633/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Thu Jun 16 17:51:08 2022] PUT /identity/v3/projects/ca560e4628b041ee8e8132b2f35b4c4c/users/efaf16b0733b426da91c5f5ae08deaac/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:51:11.069019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed02ccd-3bef-4bea-8345-56926b4052b8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.069581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed02ccd-3bef-4bea-8345-56926b4052b8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.069581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed02ccd-3bef-4bea-8345-56926b4052b8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.070945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 641/1274] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:11.077219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.077617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.077780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.094600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:11.095825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:11.161351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 634/1275] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:51:11.185109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed02ccd-3bef-4bea-8345-56926b4052b8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.287871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed02ccd-3bef-4bea-8345-56926b4052b8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.304808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:11.329137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.329137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.329137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.329137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.333506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:51:11.341107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:11.342522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:11.343431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 642/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:11.348831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.431456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:11.449157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828c461-2c62-4e63-96cb-73177a6d8cc3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.481764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.482484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.482771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.483051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.483822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:11.484134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.485486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.493310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 635/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:11.500121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.602798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d4885-48cb-4a02-ba16-1b6b2e412ff0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.636849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.637670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.637951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.638223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.649806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.651225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:51:11.668591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:11.670298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:11.671273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:51:11.671273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:51:11.671273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:11.671273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:11.671273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 643/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:11.676775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.776245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903574f-4b74-4005-9bf7-262190a64df5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.810001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.811203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.811518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.811799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.812515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:11.812830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.814185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.825527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 636/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:11.831298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:11.861831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:11.932151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e1ba-ddc4-4ef3-9a35-26722906f115 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:11.965585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:11.966863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:11.967409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:11.967548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:11.974675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:11.976258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:51:11.990663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:11.991814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:11.993020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 644/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:11.997952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.001811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.099050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3d00f-da61-44ec-ac9d-ecc0f29e14fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.133077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.133673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.133954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.134230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.135048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:12.135358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.136820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.153708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 637/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Thu Jun 16 17:51:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:12.159928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.252417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.262109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a67e0c-d6b4-48e3-83a8-8bee020d0623 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.296207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.296855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.297145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.297427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.367456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.368848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.455727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.465890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 645/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:51:12.472376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.533409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.572688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375c00c-6b7d-41dd-beca-f13e98ad14dc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.606572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.607596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.607891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.608168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.617883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.619300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:51:12.636504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:12.637594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:12.639409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:51:12.639409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:51:12.639409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:51:12.639409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:12.639409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 638/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:12.644183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.754138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad4a8a-34a8-4625-9b9a-d930b60f1a4b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.787945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.788598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.788876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.789204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.789911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:12.790213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.791601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.802870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 646/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:12.808481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:12.906503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb616-92ab-4b27-8e0f-9462b4b84b6e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:12.920545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:12.940298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:12.941077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:12.941372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:12.941714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:12.948119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:12.949539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:51:12.967904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:12.969214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 16 17:51:12.970517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 639/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:12.975125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:13.030607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:13.081865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4624373-bf4d-47c4-a85f-3e06c3fc35df None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:13.114139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:13.114785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:13.115066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:13.115340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:13.116138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:13.116442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:13.117885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d44c73-df04-409d-83ea-474f2fca9195 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:13.131120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 647/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Thu Jun 16 17:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:51:13.137177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:13.236666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c61e8f-6fd9-4b88-a40e-ef9fa28d236e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:13.271982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:13.272636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:13.272920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:13.273198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:13.274042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] RBAC: Adding query filter params (role.id=d1f5f979450b4d00b3d1a2189096884d, scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c, user.id=efaf16b0733b426da91c5f5ae08deaac) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:13.274353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:13.275805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72eb76-37a3-42d0-809e-a863f962a985 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:13.281786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 640/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Thu Jun 16 17:51:13 2022] GET /identity/v3/role_assignments?user.id=efaf16b0733b426da91c5f5ae08deaac&scope.project.id=ca560e4628b041ee8e8132b2f35b4c4c&role.id=d1f5f979450b4d00b3d1a2189096884d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:15.463323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567505a-9a2e-43c5-86f2-e2a26d4bfd1f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:15.463880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567505a-9a2e-43c5-86f2-e2a26d4bfd1f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:15.463990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567505a-9a2e-43c5-86f2-e2a26d4bfd1f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:15.465296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 648/1288] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:15.471160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:15.471667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:15.471790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:15.488207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:15.489467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:15.519058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:15.556246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 641/1289] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:51:15.578981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567505a-9a2e-43c5-86f2-e2a26d4bfd1f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:15.683266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567505a-9a2e-43c5-86f2-e2a26d4bfd1f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:15.694687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:15.719409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:15.720207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:15.720488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:15.720758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:15.721400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:15.722756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:15.734524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:15.735774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:15.736860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 649/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:51:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:15.741822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:15.839314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791558f-c59d-481b-948f-31b0215ddb5e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:15.872826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:15.873616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:15.873904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:15.874181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:15.874879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:15.875242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:15.876633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568db6f-d3e7-44cd-b2f4-b2377bc3d0c6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:15.881352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 642/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jun 16 17:51:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:15.887318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:15.986025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c164-34de-493e-bbaa-d2685a4b9ff1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:16.021777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:16.022466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:16.022750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:16.023030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:16.023780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:16.024094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:16.025514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c537a-00f0-4088-8f29-05873c51c553 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:16.030942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 650/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jun 16 17:51:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:18.137034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ebf53-701d-4920-9afd-763bc3fb0cda None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:18.138142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ebf53-701d-4920-9afd-763bc3fb0cda None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:18.138142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ebf53-701d-4920-9afd-763bc3fb0cda None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:18.139566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 643/1293] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:18.144714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae53d02-a8a0-417a-85b2-312ce0a0d95a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:18.145158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae53d02-a8a0-417a-85b2-312ce0a0d95a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:18.145276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae53d02-a8a0-417a-85b2-312ce0a0d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:18.162446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dae53d02-a8a0-417a-85b2-312ce0a0d95a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:18.163659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dae53d02-a8a0-417a-85b2-312ce0a0d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:18.193014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:18.229063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 651/1294] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:51:18.252654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ebf53-701d-4920-9afd-763bc3fb0cda None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:18.283578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:18.352571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ebf53-701d-4920-9afd-763bc3fb0cda None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:18.385656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1220f8-4efb-4e14-8cae-2b0c68f908cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:18.386309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1220f8-4efb-4e14-8cae-2b0c68f908cb None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:18.386592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1220f8-4efb-4e14-8cae-2b0c68f908cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:18.386866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1220f8-4efb-4e14-8cae-2b0c68f908cb None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:18.387691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1220f8-4efb-4e14-8cae-2b0c68f908cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:18.388984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1220f8-4efb-4e14-8cae-2b0c68f908cb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:18.498611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 644/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Thu Jun 16 17:51:18 2022] POST /identity/v3/services => generated 246 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:20.655694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c035c-f544-4143-bba5-6320aed8d604 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.656200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c035c-f544-4143-bba5-6320aed8d604 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.656200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c035c-f544-4143-bba5-6320aed8d604 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.657553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 652/1296] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:20.663074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.663543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.663668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.680455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:20.681631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:20.754356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 645/1297] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:51:20.778004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c035c-f544-4143-bba5-6320aed8d604 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:20.879795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c035c-f544-4143-bba5-6320aed8d604 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:20.913008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:20.913739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:20.914021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:20.914293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:20.914950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:20.916342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:20.928691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:20.931695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:20.932759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 653/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:51:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:51:20.936646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:20.954966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:21.037351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f6bbd-9190-43e3-9a64-7fc1ac75d4ea None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.071307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.071996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.072280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.072555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.073257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:21.073564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.074919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.080641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 646/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jun 16 17:51:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:21.086671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.122074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:21.189519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc06aa-d371-4709-a98e-716af8e983a4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.222902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.223586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.223873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.224154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.224942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] RBAC: Adding query filter params (interface=public, service_id=9618f541ccb541f7b93b53a29bf0427d, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:21.225255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.226619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4f429-c61b-4dd9-8efe-afea2cebb463 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.231713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 654/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Thu Jun 16 17:51:21 2022] GET /identity/v3/endpoints?service_id=9618f541ccb541f7b93b53a29bf0427d&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:21.237591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:21.317801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:21.335750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557cd83-f9d0-4f9c-8f70-8bae592b7d15 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:21.369005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e19f6b-6132-46bf-8a36-488795ba3012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:21.369666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e19f6b-6132-46bf-8a36-488795ba3012 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:21.369955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e19f6b-6132-46bf-8a36-488795ba3012 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:21.370235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e19f6b-6132-46bf-8a36-488795ba3012 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:21.370921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e19f6b-6132-46bf-8a36-488795ba3012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:21.372939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e19f6b-6132-46bf-8a36-488795ba3012 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:21.380193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 647/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Thu Jun 16 17:51:21 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:51:23.557349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a01c4f-c390-40af-8629-013ea1a5c9ba None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:23.557858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a01c4f-c390-40af-8629-013ea1a5c9ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:23.557858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a01c4f-c390-40af-8629-013ea1a5c9ba None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:23.560431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 655/1302] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:51:23.565232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:23.565647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:23.565800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:23.581820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:51:23.587852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:51:23.649363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 648/1303] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:51:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:51:23.673554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a01c4f-c390-40af-8629-013ea1a5c9ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:23.767442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:23.773918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a01c4f-c390-40af-8629-013ea1a5c9ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:23.807191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:23.811169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:23.811169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:23.811169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:23.811169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:23.811169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:51:23.822271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:51:23.823437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jun 16 17:51:23.824497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 656/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Thu Jun 16 17:51:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:51:23.829777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:23.905573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:23.928661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fef345-c9e0-4202-9c10-2df0c8f5210c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:23.962188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:23.962848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:23.963133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:23.963436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:23.964141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:51:23.964452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:23.966312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1035b8-2330-4c66-a50e-2bb22582d0d8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:23.971398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 649/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Thu Jun 16 17:51:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:51:23.977268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:51:24.075976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5d350-e61e-45f4-b4fb-6da6fd1464a9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:51:24.109443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d91da9-001e-48ad-a1bd-fb940afd2faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:51:24.110096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d91da9-001e-48ad-a1bd-fb940afd2faa None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:51:24.110386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d91da9-001e-48ad-a1bd-fb940afd2faa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:51:24.110666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d91da9-001e-48ad-a1bd-fb940afd2faa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:51:24.111417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d91da9-001e-48ad-a1bd-fb940afd2faa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:51:24.112777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d91da9-001e-48ad-a1bd-fb940afd2faa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:51:24.217321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 657/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Thu Jun 16 17:51:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:51:33.968346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=30 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:51:34.212086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:54.986548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3123d-cea7-427f-8441-c51f381b8294 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:54.987249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3123d-cea7-427f-8441-c51f381b8294 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:54.987249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3123d-cea7-427f-8441-c51f381b8294 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:54.989534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 650/1307] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:57:54.995391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac5af7-a3fd-498b-926a-b7209317a451 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:54.995391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac5af7-a3fd-498b-926a-b7209317a451 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:54.998796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac5af7-a3fd-498b-926a-b7209317a451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:55.084497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-83ac5af7-a3fd-498b-926a-b7209317a451 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:57:55.087310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-83ac5af7-a3fd-498b-926a-b7209317a451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:55.231272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 658/1308] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:57:55.270947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae4c1d-d9b7-4ef5-820e-1aa800732420 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:55.271242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae4c1d-d9b7-4ef5-820e-1aa800732420 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:55.271548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae4c1d-d9b7-4ef5-820e-1aa800732420 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:55.273000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 651/1309] 10.222.0.58 () {54 vars in 964 bytes} [Thu Jun 16 17:57:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:57:55.279447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bd898-d98c-493d-ac01-71c602700765 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:55.279824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bd898-d98c-493d-ac01-71c602700765 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:55.280082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bd898-d98c-493d-ac01-71c602700765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:55.371902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0c6bd898-d98c-493d-ac01-71c602700765 None None] MFA Rules not processed for user `6aae965af8284363820254c9286adf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:57:55.373333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c6bd898-d98c-493d-ac01-71c602700765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:55.398823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:55.474754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 659/1310] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Jun 16 17:57:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:57:55.479248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae4c1d-d9b7-4ef5-820e-1aa800732420 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:57:55.481470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3dae4c1d-d9b7-4ef5-820e-1aa800732420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:55.615961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae4c1d-d9b7-4ef5-820e-1aa800732420 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:57:55.704910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:57:55.705652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:55.705998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:55.706334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:55.709258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:55.770202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:55.828429 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:57:55.829852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bed1f-0aab-4a8c-acf8-ae4a1747a8dd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:57:55.850945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 652/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:57:58.315927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba00d7-43c3-43b4-be63-896c309449fb None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:58.316737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba00d7-43c3-43b4-be63-896c309449fb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:58.316737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba00d7-43c3-43b4-be63-896c309449fb None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:58.317919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 660/1312] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:57:58.323333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5cba-9a43-452a-9b26-5752bb0d9282 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:58.323763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5cba-9a43-452a-9b26-5752bb0d9282 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:58.323925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5cba-9a43-452a-9b26-5752bb0d9282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:58.407239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a59a5cba-9a43-452a-9b26-5752bb0d9282 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:57:58.408757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a59a5cba-9a43-452a-9b26-5752bb0d9282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:58.510327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 653/1313] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:57:58.566313 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d2105-786a-45fc-ae3a-aeb9a79fd91e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:58.566786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d2105-786a-45fc-ae3a-aeb9a79fd91e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:58.567162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d2105-786a-45fc-ae3a-aeb9a79fd91e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:58.568918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 661/1314] 10.222.0.58 () {54 vars in 964 bytes} [Thu Jun 16 17:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:57:58.575288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315e91c-10d1-44b3-8be8-b27490bdb5d0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:58.575709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315e91c-10d1-44b3-8be8-b27490bdb5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:58.576000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315e91c-10d1-44b3-8be8-b27490bdb5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:58.586993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:58.658122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3315e91c-10d1-44b3-8be8-b27490bdb5d0 None None] MFA Rules not processed for user `6aae965af8284363820254c9286adf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:57:58.660002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3315e91c-10d1-44b3-8be8-b27490bdb5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:58.740184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 654/1315] 10.222.0.58 () {58 vars in 1061 bytes} [Thu Jun 16 17:57:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:57:58.749210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d2105-786a-45fc-ae3a-aeb9a79fd91e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:57:58.750724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-943d2105-786a-45fc-ae3a-aeb9a79fd91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:58.866021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d2105-786a-45fc-ae3a-aeb9a79fd91e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:57:58.936787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:57:58.939229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:57:58.939902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:57:58.940183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:57:58.940456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:57:58.942665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:57:59.053830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:57:59.055273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ec520-1fcd-4189-bfc4-269b718d934a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:57:59.073796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 662/1316] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:02.015358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ba04a-a9c3-48c8-9026-a7277dc3bf60 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:02.015911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ba04a-a9c3-48c8-9026-a7277dc3bf60 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:02.016009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ba04a-a9c3-48c8-9026-a7277dc3bf60 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:02.017317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 655/1317] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:02.022490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c3422-6221-4fff-96b2-7e4cdf0c9ad5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:02.022877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c3422-6221-4fff-96b2-7e4cdf0c9ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:02.023046 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c3422-6221-4fff-96b2-7e4cdf0c9ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:02.086590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-967c3422-6221-4fff-96b2-7e4cdf0c9ad5 None None] MFA Rules not processed for user `b1b8dc4aab034096af0d6b9ede657d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:02.087829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-967c3422-6221-4fff-96b2-7e4cdf0c9ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:02.139924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:02.185720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 663/1318] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:02.226055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ba04a-a9c3-48c8-9026-a7277dc3bf60 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:02.227842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-797ba04a-a9c3-48c8-9026-a7277dc3bf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:02.334352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ba04a-a9c3-48c8-9026-a7277dc3bf60 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:02.337127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:02.410213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:02.410864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:02.411151 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:02.411491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:02.413150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:02.521724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:02.523188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36d0b4-6fac-43f1-8cf2-773f0540eeb2 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:02.542063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 656/1319] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:05.592424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263e356-74f4-4e02-9898-1db98f8d0026 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:05.592956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263e356-74f4-4e02-9898-1db98f8d0026 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:05.592956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263e356-74f4-4e02-9898-1db98f8d0026 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:05.594277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 664/1320] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:05.600043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b9e3-6e09-4eea-b37f-ad67a968cd09 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:05.600472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b9e3-6e09-4eea-b37f-ad67a968cd09 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:05.600585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b9e3-6e09-4eea-b37f-ad67a968cd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:05.640378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:05.665674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8809b9e3-6e09-4eea-b37f-ad67a968cd09 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:05.666863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8809b9e3-6e09-4eea-b37f-ad67a968cd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:05.763458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 657/1321] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:05.802587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263e356-74f4-4e02-9898-1db98f8d0026 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:05.804148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2263e356-74f4-4e02-9898-1db98f8d0026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:05.905914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263e356-74f4-4e02-9898-1db98f8d0026 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:05.976734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:05.977384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:05.977669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:05.977947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:05.979601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:05.986779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:06.085916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:06.087348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e7636-f431-4493-8d56-e968bb810f40 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:06.104860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 665/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:09.345846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd582adc-4424-403a-8992-edd28fe313e3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:09.346407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd582adc-4424-403a-8992-edd28fe313e3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:09.346407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd582adc-4424-403a-8992-edd28fe313e3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:09.347869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 658/1323] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:09.353370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c8755-2bb6-4a9a-a2de-d2317fe2894a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:09.353765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c8755-2bb6-4a9a-a2de-d2317fe2894a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:09.353932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c8755-2bb6-4a9a-a2de-d2317fe2894a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:09.419103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8e3c8755-2bb6-4a9a-a2de-d2317fe2894a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:09.420377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8e3c8755-2bb6-4a9a-a2de-d2317fe2894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:09.480529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:09.530709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 666/1324] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:09.568960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd582adc-4424-403a-8992-edd28fe313e3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:09.570526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fd582adc-4424-403a-8992-edd28fe313e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:09.684829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd582adc-4424-403a-8992-edd28fe313e3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:09.697964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:09.763823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:09.764485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:09.764765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:09.765045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:09.766710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:09.889296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:09.890642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fee0ad-d0f9-4d53-b626-266b6b6cc73b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:09.913478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 659/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:12.137931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ab820-e60d-4464-baaf-afc0357f737a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:12.138472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ab820-e60d-4464-baaf-afc0357f737a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:12.138472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ab820-e60d-4464-baaf-afc0357f737a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:12.139827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 667/1326] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:12.145238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ed91-3503-413b-867f-ba43ef1f45dd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:12.145680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ed91-3503-413b-867f-ba43ef1f45dd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:12.145791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3ed91-3503-413b-867f-ba43ef1f45dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:12.192943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:12.212241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5ca3ed91-3503-413b-867f-ba43ef1f45dd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:12.213525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5ca3ed91-3503-413b-867f-ba43ef1f45dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:12.315244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 660/1327] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:12.351007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ab820-e60d-4464-baaf-afc0357f737a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:12.352667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fe1ab820-e60d-4464-baaf-afc0357f737a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:12.460248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ab820-e60d-4464-baaf-afc0357f737a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:12.535119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:12.535810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:12.536095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:12.536378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:12.538054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:12.556380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:12.653299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:12.654740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a52353-2cba-4b65-9d53-a3cb7e76a0f9 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:12.673568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 668/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:14.895159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e3f8d-0c77-4938-89f3-027e7c123aa2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:14.895691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e3f8d-0c77-4938-89f3-027e7c123aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:14.895691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e3f8d-0c77-4938-89f3-027e7c123aa2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:14.897273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 661/1329] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:14.902682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71020d2f-2285-4672-94e6-edbb4951782c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:14.903209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71020d2f-2285-4672-94e6-edbb4951782c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:14.903295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71020d2f-2285-4672-94e6-edbb4951782c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:14.974061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-71020d2f-2285-4672-94e6-edbb4951782c None None] MFA Rules not processed for user `b1b8dc4aab034096af0d6b9ede657d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:14.975308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-71020d2f-2285-4672-94e6-edbb4951782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:15.045902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:15.087018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 669/1330] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:15.127938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e3f8d-0c77-4938-89f3-027e7c123aa2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:15.129502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-421e3f8d-0c77-4938-89f3-027e7c123aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:15.247858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e3f8d-0c77-4938-89f3-027e7c123aa2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:15.271585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:15.328993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:15.329695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:15.330063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:15.330346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:15.332123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:15.456435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:15.457882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30de86-cbc8-4795-b163-d90128616e8f service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:15.477011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 662/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:18.271131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7343a9-0e7f-456e-b781-d8fafeef982c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:18.271672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7343a9-0e7f-456e-b781-d8fafeef982c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:18.271775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7343a9-0e7f-456e-b781-d8fafeef982c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:18.273133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 670/1332] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:18.278259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1568-ab16-45c5-af9e-6a7ac977335d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:18.281395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1568-ab16-45c5-af9e-6a7ac977335d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:18.281395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1568-ab16-45c5-af9e-6a7ac977335d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:18.340892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:18.343575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5edd1568-ab16-45c5-af9e-6a7ac977335d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:18.344804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5edd1568-ab16-45c5-af9e-6a7ac977335d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:18.441296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 663/1333] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:18.477011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7343a9-0e7f-456e-b781-d8fafeef982c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:18.478547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2e7343a9-0e7f-456e-b781-d8fafeef982c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:18.580524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7343a9-0e7f-456e-b781-d8fafeef982c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:18.653379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:18.654031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:18.654315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:18.654590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:18.656277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:18.683350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:18.764115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:18.765534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764b7d6-0419-431f-a773-3d249ad7d091 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:18.784716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 671/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:20.998742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf01c49-c723-45d8-8ba8-6e6e47489154 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:20.999179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf01c49-c723-45d8-8ba8-6e6e47489154 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:20.999293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf01c49-c723-45d8-8ba8-6e6e47489154 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:21.000780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 664/1335] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:21.006423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a1170-ac3d-435e-837c-94cc551b5034 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:21.006863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a1170-ac3d-435e-837c-94cc551b5034 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:21.006987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a1170-ac3d-435e-837c-94cc551b5034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:21.070463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8a2a1170-ac3d-435e-837c-94cc551b5034 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:21.071757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8a2a1170-ac3d-435e-837c-94cc551b5034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:21.144800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:21.165901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 672/1336] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:21.206496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf01c49-c723-45d8-8ba8-6e6e47489154 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:21.208071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-caf01c49-c723-45d8-8ba8-6e6e47489154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:21.314122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf01c49-c723-45d8-8ba8-6e6e47489154 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:21.341628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:21.390329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:21.390971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:21.391250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:21.392115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:21.393773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:21.505699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:21.507032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da22940-2b00-499f-9ff7-1f59a5f5f025 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:21.525422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 665/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:58:24.382069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b6bb0-8a43-4f81-b73c-4b0095959a56 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:24.382643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b6bb0-8a43-4f81-b73c-4b0095959a56 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:24.382643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b6bb0-8a43-4f81-b73c-4b0095959a56 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:24.384013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 673/1338] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:24.389415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f01c4-45eb-4a61-8a35-46cfb3a64a38 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:24.389842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f01c4-45eb-4a61-8a35-46cfb3a64a38 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:24.389993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f01c4-45eb-4a61-8a35-46cfb3a64a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:24.454967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-973f01c4-45eb-4a61-8a35-46cfb3a64a38 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:24.456119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-973f01c4-45eb-4a61-8a35-46cfb3a64a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:24.464372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:24.553238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 666/1339] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:24.589725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b6bb0-8a43-4f81-b73c-4b0095959a56 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:24.591269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-603b6bb0-8a43-4f81-b73c-4b0095959a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:24.696310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b6bb0-8a43-4f81-b73c-4b0095959a56 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:24.770054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:24.770624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:24.770912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:24.771192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:24.772890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:24.808462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:24.882284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:24.883776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3e501-2284-4e88-a2b3-018629cc60fa service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:24.901594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 674/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:29.379557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc18983-da56-4d19-9c25-bb4cc60ace79 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:29.380114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc18983-da56-4d19-9c25-bb4cc60ace79 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:29.380267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc18983-da56-4d19-9c25-bb4cc60ace79 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:29.381743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 667/1341] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:29.387427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78591a-3340-425d-9b61-79c32392b730 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:29.387866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78591a-3340-425d-9b61-79c32392b730 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:29.388113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78591a-3340-425d-9b61-79c32392b730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:29.473678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3e78591a-3340-425d-9b61-79c32392b730 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:29.474902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3e78591a-3340-425d-9b61-79c32392b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:29.567887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:29.580601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 675/1342] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:29.620093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc18983-da56-4d19-9c25-bb4cc60ace79 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:29.622218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9dc18983-da56-4d19-9c25-bb4cc60ace79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:29.718097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc18983-da56-4d19-9c25-bb4cc60ace79 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:29.722353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6d31c-3d01-4526-9629-5a5309464ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:29.722999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d31c-3d01-4526-9629-5a5309464ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:29.723284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d31c-3d01-4526-9629-5a5309464ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:29.723590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d31c-3d01-4526-9629-5a5309464ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:29.733190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6d31c-3d01-4526-9629-5a5309464ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:29.735337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6d31c-3d01-4526-9629-5a5309464ee7 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:29.745090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 668/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:32.990108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66952e-0903-42c9-bd47-605616da0b06 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:32.990646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66952e-0903-42c9-bd47-605616da0b06 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:32.990646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66952e-0903-42c9-bd47-605616da0b06 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:32.992092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 676/1344] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:32.997213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4bfed-42be-419b-9976-540e35e4c624 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:32.997619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4bfed-42be-419b-9976-540e35e4c624 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:32.997794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4bfed-42be-419b-9976-540e35e4c624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:33.027141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c1d4bfed-42be-419b-9976-540e35e4c624 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:33.028393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c1d4bfed-42be-419b-9976-540e35e4c624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:33.056628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:33.106487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 669/1345] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:33.144400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66952e-0903-42c9-bd47-605616da0b06 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:33.154871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66952e-0903-42c9-bd47-605616da0b06 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:33.158546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7083-da34-4c14-80d5-1b672c436b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:33.159184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7083-da34-4c14-80d5-1b672c436b9a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:33.159507 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7083-da34-4c14-80d5-1b672c436b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:33.159788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7083-da34-4c14-80d5-1b672c436b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:33.168729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc7083-da34-4c14-80d5-1b672c436b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:33.170140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc7083-da34-4c14-80d5-1b672c436b9a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:33.180428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 677/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:37.647179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193d435-d100-48a3-923f-6ffe71fc0918 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:37.647914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193d435-d100-48a3-923f-6ffe71fc0918 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:37.647914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193d435-d100-48a3-923f-6ffe71fc0918 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:37.649276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 670/1347] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:37.659531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e141f-3198-4761-a858-b127c167f94a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:37.659531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e141f-3198-4761-a858-b127c167f94a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:37.659531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e141f-3198-4761-a858-b127c167f94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:37.680706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0e7e141f-3198-4761-a858-b127c167f94a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:37.682043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0e7e141f-3198-4761-a858-b127c167f94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:37.694129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 678/1348] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:37.737840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193d435-d100-48a3-923f-6ffe71fc0918 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:37.754060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193d435-d100-48a3-923f-6ffe71fc0918 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:37.758115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb3f9b-e339-41c9-93ee-2a29191f214c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:37.759155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb3f9b-e339-41c9-93ee-2a29191f214c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:37.759547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb3f9b-e339-41c9-93ee-2a29191f214c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:37.759836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb3f9b-e339-41c9-93ee-2a29191f214c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:37.771885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb3f9b-e339-41c9-93ee-2a29191f214c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:37.773563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb3f9b-e339-41c9-93ee-2a29191f214c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:37.788004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 671/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:42.961352 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac5716-f966-4ada-8a91-dac4d84632e3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:42.961900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac5716-f966-4ada-8a91-dac4d84632e3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:42.961900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac5716-f966-4ada-8a91-dac4d84632e3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:42.963274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 679/1350] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 17:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:42.968352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b7c50-e121-44ce-9929-65120653c164 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:42.969216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b7c50-e121-44ce-9929-65120653c164 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:42.969652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b7c50-e121-44ce-9929-65120653c164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:43.099451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-286b7c50-e121-44ce-9929-65120653c164 None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:43.100960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-286b7c50-e121-44ce-9929-65120653c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:43.178912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 672/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 17:58:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:47.522154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d74edb-f7da-4193-9578-58d7eea92270 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:47.523035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d74edb-f7da-4193-9578-58d7eea92270 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:47.523322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d74edb-f7da-4193-9578-58d7eea92270 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:47.524727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 680/1352] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 17:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:47.530753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed715a-7ebc-4efc-94bc-0901f14cebe5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:47.531257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed715a-7ebc-4efc-94bc-0901f14cebe5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:47.531349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed715a-7ebc-4efc-94bc-0901f14cebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:47.550682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-39ed715a-7ebc-4efc-94bc-0901f14cebe5 None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:47.552143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-39ed715a-7ebc-4efc-94bc-0901f14cebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:47.561617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 673/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 17:58:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:47.672963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=13 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:49.043220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384300be-1139-4e06-baec-8bfa16f4e415 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:49.043893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384300be-1139-4e06-baec-8bfa16f4e415 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:49.043893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384300be-1139-4e06-baec-8bfa16f4e415 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:49.046071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 681/1354] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 17:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:49.051247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f107b80-0231-4b56-998c-bc46fa2a5732 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:49.051709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f107b80-0231-4b56-998c-bc46fa2a5732 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:49.053958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f107b80-0231-4b56-998c-bc46fa2a5732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:49.071162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3f107b80-0231-4b56-998c-bc46fa2a5732 None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:49.072507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f107b80-0231-4b56-998c-bc46fa2a5732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:49.081296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 674/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 17:58:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:52.960779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030e6bd-1b41-4ea0-85f1-4955b2fb0b1c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:52.961380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030e6bd-1b41-4ea0-85f1-4955b2fb0b1c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:52.961380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030e6bd-1b41-4ea0-85f1-4955b2fb0b1c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:52.962850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 682/1356] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:52.969383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f133a-2275-4fd1-9c7a-dee1e9967a1c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:52.970207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f133a-2275-4fd1-9c7a-dee1e9967a1c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:52.970492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f133a-2275-4fd1-9c7a-dee1e9967a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:52.988722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2a3f133a-2275-4fd1-9c7a-dee1e9967a1c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:52.989919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2a3f133a-2275-4fd1-9c7a-dee1e9967a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:52.998458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 675/1357] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:53.055709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c4f90-a315-4595-b171-b2c39164a9a2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:53.056783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c4f90-a315-4595-b171-b2c39164a9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:53.057416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c4f90-a315-4595-b171-b2c39164a9a2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:53.060758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 683/1358] 10.222.0.58 () {54 vars in 961 bytes} [Thu Jun 16 17:58:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:53.069025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887284a-2680-4d72-92cb-71c031d02b9b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:53.069576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887284a-2680-4d72-92cb-71c031d02b9b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:53.070089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887284a-2680-4d72-92cb-71c031d02b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:53.132243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:58:53.140979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4887284a-2680-4d72-92cb-71c031d02b9b None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:53.142308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4887284a-2680-4d72-92cb-71c031d02b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:53.261338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 676/1359] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Jun 16 17:58:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:53.268851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c4f90-a315-4595-b171-b2c39164a9a2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:53.284160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c4f90-a315-4595-b171-b2c39164a9a2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:53.290144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844867e-6843-4100-8934-983f89cbf34d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:53.291135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844867e-6843-4100-8934-983f89cbf34d service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:53.291652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844867e-6843-4100-8934-983f89cbf34d service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:53.292153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844867e-6843-4100-8934-983f89cbf34d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:53.302057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844867e-6843-4100-8934-983f89cbf34d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:53.303906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844867e-6843-4100-8934-983f89cbf34d service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:53.315084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 684/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:55.682708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc868-1470-48bf-a0f6-cacca9ea46e0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:55.683250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc868-1470-48bf-a0f6-cacca9ea46e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:55.683250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fc868-1470-48bf-a0f6-cacca9ea46e0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:55.685526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 677/1361] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:55.691398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480d61-dc41-48f6-a6f7-363cbba7a9e6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:55.691943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480d61-dc41-48f6-a6f7-363cbba7a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:55.691943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480d61-dc41-48f6-a6f7-363cbba7a9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:55.717513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-60480d61-dc41-48f6-a6f7-363cbba7a9e6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:55.719130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-60480d61-dc41-48f6-a6f7-363cbba7a9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:55.735418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 685/1362] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:55.785468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fc868-1470-48bf-a0f6-cacca9ea46e0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:55.804120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fc868-1470-48bf-a0f6-cacca9ea46e0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:55.810621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff81f-fdf7-47f7-b1e4-b360a04305c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:55.811571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff81f-fdf7-47f7-b1e4-b360a04305c8 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:55.811865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff81f-fdf7-47f7-b1e4-b360a04305c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:55.812148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff81f-fdf7-47f7-b1e4-b360a04305c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:55.821019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ff81f-fdf7-47f7-b1e4-b360a04305c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:55.822593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ff81f-fdf7-47f7-b1e4-b360a04305c8 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:55.832817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 678/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:58:58.514392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140c8f8-1ced-40b8-b3de-2f63e3cf1ca5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:58.514880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140c8f8-1ced-40b8-b3de-2f63e3cf1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:58.514880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140c8f8-1ced-40b8-b3de-2f63e3cf1ca5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:58.516358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 686/1364] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:58.521607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350f5ab-eefb-401f-8a13-a2b2e343bd21 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:58.522075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350f5ab-eefb-401f-8a13-a2b2e343bd21 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:58.522185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350f5ab-eefb-401f-8a13-a2b2e343bd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:58.548837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a350f5ab-eefb-401f-8a13-a2b2e343bd21 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:58.550054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a350f5ab-eefb-401f-8a13-a2b2e343bd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:58.558008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 679/1365] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:58:58.607335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52972-dcf1-4d45-966c-9d978c2f4517 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:58.607763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52972-dcf1-4d45-966c-9d978c2f4517 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:58.608190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb52972-dcf1-4d45-966c-9d978c2f4517 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:58.609642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 687/1366] 10.222.0.58 () {54 vars in 961 bytes} [Thu Jun 16 17:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:58:58.615160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127972-ae04-4a29-a2ce-5ef77dc798ce None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:58.615660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127972-ae04-4a29-a2ce-5ef77dc798ce None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:58.615945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127972-ae04-4a29-a2ce-5ef77dc798ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:58.635984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5b127972-ae04-4a29-a2ce-5ef77dc798ce None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:58:58.637203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5b127972-ae04-4a29-a2ce-5ef77dc798ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:58:58.646650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 680/1367] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Jun 16 17:58:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:58:58.654611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52972-dcf1-4d45-966c-9d978c2f4517 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:58:58.671923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb52972-dcf1-4d45-966c-9d978c2f4517 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:58:58.678992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d808e-699d-4bf6-adc6-490d0493b24f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:58:58.684153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d808e-699d-4bf6-adc6-490d0493b24f service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:58:58.685288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d808e-699d-4bf6-adc6-490d0493b24f service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:58:58.685738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d808e-699d-4bf6-adc6-490d0493b24f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:58:58.702682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d808e-699d-4bf6-adc6-490d0493b24f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:58:58.704653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d808e-699d-4bf6-adc6-490d0493b24f service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:58:58.716715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 688/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:01.101801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6192d2-40dc-4ba1-9570-38224fb9ea7b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.102323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6192d2-40dc-4ba1-9570-38224fb9ea7b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.102323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6192d2-40dc-4ba1-9570-38224fb9ea7b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.103821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 681/1369] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:01.110074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0450eaf-5b9b-45e0-aa19-5b54d5754be1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.110627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0450eaf-5b9b-45e0-aa19-5b54d5754be1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.110627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0450eaf-5b9b-45e0-aa19-5b54d5754be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.129447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d0450eaf-5b9b-45e0-aa19-5b54d5754be1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:01.130671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d0450eaf-5b9b-45e0-aa19-5b54d5754be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:01.139525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 689/1370] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:01.183894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6192d2-40dc-4ba1-9570-38224fb9ea7b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:01.201809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6192d2-40dc-4ba1-9570-38224fb9ea7b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:01.207884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c17b5-6da4-4a29-b8a3-5b2d4d6ec2be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:01.208677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c17b5-6da4-4a29-b8a3-5b2d4d6ec2be service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.209003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c17b5-6da4-4a29-b8a3-5b2d4d6ec2be service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.209285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c17b5-6da4-4a29-b8a3-5b2d4d6ec2be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.218881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c17b5-6da4-4a29-b8a3-5b2d4d6ec2be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:01.220487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c17b5-6da4-4a29-b8a3-5b2d4d6ec2be service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:01.230342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 682/1371] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:01.382930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7c3f1-36af-48e8-8f3a-a434294e3100 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.383281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7c3f1-36af-48e8-8f3a-a434294e3100 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.383646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7c3f1-36af-48e8-8f3a-a434294e3100 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.385011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 690/1372] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:01.390403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f28e-b826-406b-853e-461bfd0854a6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.390702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f28e-b826-406b-853e-461bfd0854a6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.392976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f28e-b826-406b-853e-461bfd0854a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.409055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c2d7f28e-b826-406b-853e-461bfd0854a6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:01.410272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2d7f28e-b826-406b-853e-461bfd0854a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:01.418259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 683/1373] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:01.737846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62879a-6cf2-4591-9926-c5b500e2a9fb None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.738138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62879a-6cf2-4591-9926-c5b500e2a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.738420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62879a-6cf2-4591-9926-c5b500e2a9fb None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.742389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 691/1374] 10.222.0.58 () {54 vars in 963 bytes} [Thu Jun 16 17:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:01.744919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a39df-ad82-41b4-88be-486afaaee227 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.745319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a39df-ad82-41b4-88be-486afaaee227 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.745703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a39df-ad82-41b4-88be-486afaaee227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.801960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-091a39df-ad82-41b4-88be-486afaaee227 None None] MFA Rules not processed for user `fbc41720695b45ec84c2f250238ac37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:01.803543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-091a39df-ad82-41b4-88be-486afaaee227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:01.876536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 684/1375] 10.222.0.58 () {58 vars in 1060 bytes} [Thu Jun 16 17:59:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:01.884509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62879a-6cf2-4591-9926-c5b500e2a9fb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:01.896826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62879a-6cf2-4591-9926-c5b500e2a9fb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:01.900704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb949b-34f3-485a-8f6d-1a98e31c60c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:01.901354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb949b-34f3-485a-8f6d-1a98e31c60c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:01.901843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb949b-34f3-485a-8f6d-1a98e31c60c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:01.902282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb949b-34f3-485a-8f6d-1a98e31c60c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:01.912903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb949b-34f3-485a-8f6d-1a98e31c60c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:01.914609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb949b-34f3-485a-8f6d-1a98e31c60c7 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:01.925992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 692/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:03.614989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1341d-54a8-4fbd-97f2-ce1d7c03dbfd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:03.615563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1341d-54a8-4fbd-97f2-ce1d7c03dbfd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:03.615563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1341d-54a8-4fbd-97f2-ce1d7c03dbfd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:03.616994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 685/1377] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:59:03.623882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffdb37-7c40-48c0-9aad-f840392d8fed None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:03.624426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffdb37-7c40-48c0-9aad-f840392d8fed None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:03.624426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffdb37-7c40-48c0-9aad-f840392d8fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:03.643804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-03ffdb37-7c40-48c0-9aad-f840392d8fed None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:03.645034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-03ffdb37-7c40-48c0-9aad-f840392d8fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:03.653468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 693/1378] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:03.701238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1341d-54a8-4fbd-97f2-ce1d7c03dbfd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:03.716487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1341d-54a8-4fbd-97f2-ce1d7c03dbfd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:03.723796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898fef-60f0-496e-9287-48750bd06167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:03.723796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898fef-60f0-496e-9287-48750bd06167 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:03.723796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898fef-60f0-496e-9287-48750bd06167 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:03.723796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898fef-60f0-496e-9287-48750bd06167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:03.732631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4898fef-60f0-496e-9287-48750bd06167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:03.734435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4898fef-60f0-496e-9287-48750bd06167 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:03.738822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:03.745214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 686/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:06.102922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0b61d-653a-432e-b478-a58e38ba0f06 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:06.103484 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0b61d-653a-432e-b478-a58e38ba0f06 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:06.103601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0b61d-653a-432e-b478-a58e38ba0f06 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:06.105038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 694/1380] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:06.110629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c24da-4907-41d8-afe7-ce9dbc0e0f46 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:06.111095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c24da-4907-41d8-afe7-ce9dbc0e0f46 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:06.111190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c24da-4907-41d8-afe7-ce9dbc0e0f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:06.130261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-843c24da-4907-41d8-afe7-ce9dbc0e0f46 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:06.131486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-843c24da-4907-41d8-afe7-ce9dbc0e0f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:06.140048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 687/1381] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:06.184605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0b61d-653a-432e-b478-a58e38ba0f06 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:06.199357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0b61d-653a-432e-b478-a58e38ba0f06 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:06.205079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d39559-fa6c-40d2-b2cb-2dd895dbf693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:06.205730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d39559-fa6c-40d2-b2cb-2dd895dbf693 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:06.206018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d39559-fa6c-40d2-b2cb-2dd895dbf693 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:06.206296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d39559-fa6c-40d2-b2cb-2dd895dbf693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:06.215040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d39559-fa6c-40d2-b2cb-2dd895dbf693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:06.216744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d39559-fa6c-40d2-b2cb-2dd895dbf693 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:06.226727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 695/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:59:08.542409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39c5de-a109-49b4-805a-d8a4e1524263 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:08.542964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39c5de-a109-49b4-805a-d8a4e1524263 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:08.542964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39c5de-a109-49b4-805a-d8a4e1524263 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:08.545476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 688/1383] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:08.553277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad02e6-f213-49ca-a200-f58fe5b295b9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:08.553763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad02e6-f213-49ca-a200-f58fe5b295b9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:08.553846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad02e6-f213-49ca-a200-f58fe5b295b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:08.573064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4dad02e6-f213-49ca-a200-f58fe5b295b9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:08.575399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4dad02e6-f213-49ca-a200-f58fe5b295b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:08.584567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 696/1384] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:08.630612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39c5de-a109-49b4-805a-d8a4e1524263 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:08.645878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39c5de-a109-49b4-805a-d8a4e1524263 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:08.651482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2d4da-6814-42b8-a4a7-7449ff4ebdae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:08.652362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d4da-6814-42b8-a4a7-7449ff4ebdae service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:08.652655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d4da-6814-42b8-a4a7-7449ff4ebdae service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:08.652937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d4da-6814-42b8-a4a7-7449ff4ebdae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:08.661670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2d4da-6814-42b8-a4a7-7449ff4ebdae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:08.663248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2d4da-6814-42b8-a4a7-7449ff4ebdae service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:08.673933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 689/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:59:10.809250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c236e-3a8b-471d-ac15-cb1db9af0844 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:10.810004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c236e-3a8b-471d-ac15-cb1db9af0844 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:10.810004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c236e-3a8b-471d-ac15-cb1db9af0844 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:10.811175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 697/1386] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:10.817181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb4f70-7bba-4deb-a6bc-3a4f6efef018 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:10.817658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb4f70-7bba-4deb-a6bc-3a4f6efef018 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:10.817904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb4f70-7bba-4deb-a6bc-3a4f6efef018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:10.841612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-33fb4f70-7bba-4deb-a6bc-3a4f6efef018 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:10.843200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-33fb4f70-7bba-4deb-a6bc-3a4f6efef018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:10.853052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 690/1387] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:10.901156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c236e-3a8b-471d-ac15-cb1db9af0844 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:10.916848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c236e-3a8b-471d-ac15-cb1db9af0844 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:10.922522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfab6b5-945e-48bf-92e6-8da17c009685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:10.923281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfab6b5-945e-48bf-92e6-8da17c009685 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:10.923609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfab6b5-945e-48bf-92e6-8da17c009685 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:10.923893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfab6b5-945e-48bf-92e6-8da17c009685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:10.932626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfab6b5-945e-48bf-92e6-8da17c009685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:10.934355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfab6b5-945e-48bf-92e6-8da17c009685 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:10.944160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 698/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:13.385611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad981ee-e7a2-413f-a9cb-824318b18f7f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:13.386286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad981ee-e7a2-413f-a9cb-824318b18f7f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:13.386286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad981ee-e7a2-413f-a9cb-824318b18f7f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:13.388086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 691/1389] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:13.393797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f920e18-1331-42f4-b5d2-a43e6b473389 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:13.394417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f920e18-1331-42f4-b5d2-a43e6b473389 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:13.394417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f920e18-1331-42f4-b5d2-a43e6b473389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:13.412974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5f920e18-1331-42f4-b5d2-a43e6b473389 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:13.414204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5f920e18-1331-42f4-b5d2-a43e6b473389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:13.422455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 699/1390] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:13.467635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad981ee-e7a2-413f-a9cb-824318b18f7f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:13.486386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad981ee-e7a2-413f-a9cb-824318b18f7f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:13.491912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db64b5-ae34-4756-8504-9c6126ab7c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:13.492563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db64b5-ae34-4756-8504-9c6126ab7c0a service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:13.492850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db64b5-ae34-4756-8504-9c6126ab7c0a service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:13.493128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db64b5-ae34-4756-8504-9c6126ab7c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:13.502276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db64b5-ae34-4756-8504-9c6126ab7c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:13.503895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db64b5-ae34-4756-8504-9c6126ab7c0a service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:13.514622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 692/1391] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:15.919826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041cabc-4f0b-4093-ae3c-bb0908390cfe None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:15.920385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041cabc-4f0b-4093-ae3c-bb0908390cfe None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:15.920385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041cabc-4f0b-4093-ae3c-bb0908390cfe None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:15.921764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 700/1392] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:59:15.927037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef84434-ff63-4139-b07f-8573a183b86a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:15.927541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef84434-ff63-4139-b07f-8573a183b86a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:15.927637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef84434-ff63-4139-b07f-8573a183b86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:15.946256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1ef84434-ff63-4139-b07f-8573a183b86a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:15.947504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ef84434-ff63-4139-b07f-8573a183b86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:15.957956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 693/1393] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:16.003869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041cabc-4f0b-4093-ae3c-bb0908390cfe None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:16.018634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041cabc-4f0b-4093-ae3c-bb0908390cfe None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:16.024118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955a27f-eb05-491e-afc3-e2afe6b6fffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:16.024762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a27f-eb05-491e-afc3-e2afe6b6fffc service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:16.025052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a27f-eb05-491e-afc3-e2afe6b6fffc service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:16.025334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955a27f-eb05-491e-afc3-e2afe6b6fffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:16.033977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955a27f-eb05-491e-afc3-e2afe6b6fffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:16.035518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955a27f-eb05-491e-afc3-e2afe6b6fffc service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:16.045765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 701/1394] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:17.923089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a3a30-be3b-4b71-8955-7dad583f9163 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:17.923736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a3a30-be3b-4b71-8955-7dad583f9163 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:17.923736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a3a30-be3b-4b71-8955-7dad583f9163 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:17.925058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 694/1395] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:17.930372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8a98c-d747-43ca-8c85-3c30532bb6f8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:17.930795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8a98c-d747-43ca-8c85-3c30532bb6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:17.930937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8a98c-d747-43ca-8c85-3c30532bb6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:17.937638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=52 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:17.950915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c8f8a98c-d747-43ca-8c85-3c30532bb6f8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:17.954009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c8f8a98c-d747-43ca-8c85-3c30532bb6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:17.960130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 702/1396] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:18.019491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dde57-d583-4f99-b5ac-a94558928739 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:18.019860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dde57-d583-4f99-b5ac-a94558928739 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:18.020214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dde57-d583-4f99-b5ac-a94558928739 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:18.021674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 695/1397] 10.222.0.58 () {54 vars in 962 bytes} [Thu Jun 16 17:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:18.027024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4977daf-407b-431b-a6da-cd07a418a994 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:18.027433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4977daf-407b-431b-a6da-cd07a418a994 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:18.027797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4977daf-407b-431b-a6da-cd07a418a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:18.078270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b4977daf-407b-431b-a6da-cd07a418a994 None None] MFA Rules not processed for user `6f37017740874fef845f23fca4f977f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:18.079503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b4977daf-407b-431b-a6da-cd07a418a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:18.138805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 703/1398] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Jun 16 17:59:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:18.146205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dde57-d583-4f99-b5ac-a94558928739 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:18.157066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dde57-d583-4f99-b5ac-a94558928739 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:18.160547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f7acd-6143-46ac-804a-289731893aeb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:18.161247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7acd-6143-46ac-804a-289731893aeb service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:18.161598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7acd-6143-46ac-804a-289731893aeb service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:18.161962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f7acd-6143-46ac-804a-289731893aeb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:18.171942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7acd-6143-46ac-804a-289731893aeb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:18.173504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f7acd-6143-46ac-804a-289731893aeb service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:18.183856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 696/1399] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:18.364177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4ee9-ea7a-4e72-88c9-226f526d1ef2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:18.364466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4ee9-ea7a-4e72-88c9-226f526d1ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:18.364752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c4ee9-ea7a-4e72-88c9-226f526d1ef2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:18.366203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 704/1400] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:18.371539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714f5bb-6332-4c31-860b-47d571caea7a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:18.371833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714f5bb-6332-4c31-860b-47d571caea7a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:18.372109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714f5bb-6332-4c31-860b-47d571caea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:18.390797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4714f5bb-6332-4c31-860b-47d571caea7a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:18.392019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4714f5bb-6332-4c31-860b-47d571caea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:18.399932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 697/1401] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:18.445704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4ee9-ea7a-4e72-88c9-226f526d1ef2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:18.460312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c4ee9-ea7a-4e72-88c9-226f526d1ef2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:18.466459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e08a0-6dc9-44fb-9696-a667b291ff8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:18.467110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e08a0-6dc9-44fb-9696-a667b291ff8e service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:18.467426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e08a0-6dc9-44fb-9696-a667b291ff8e service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:18.467706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e08a0-6dc9-44fb-9696-a667b291ff8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:18.476044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e08a0-6dc9-44fb-9696-a667b291ff8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:18.477605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e08a0-6dc9-44fb-9696-a667b291ff8e service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:18.487769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 705/1402] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:19.137171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff39d0-e9fe-42ee-af8e-3f72b1f2aa6f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:19.137616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff39d0-e9fe-42ee-af8e-3f72b1f2aa6f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:19.138023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff39d0-e9fe-42ee-af8e-3f72b1f2aa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:19.188173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-39ff39d0-e9fe-42ee-af8e-3f72b1f2aa6f None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:19.189606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-39ff39d0-e9fe-42ee-af8e-3f72b1f2aa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:19.258106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 698/1403] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 17:59:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:19.304394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89372a-f393-4253-9f00-6a13bb4d2c4d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:19.304869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89372a-f393-4253-9f00-6a13bb4d2c4d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:19.305153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89372a-f393-4253-9f00-6a13bb4d2c4d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:19.306641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 706/1404] 10.222.0.58 () {54 vars in 942 bytes} [Thu Jun 16 17:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:19.313034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ce59e-9f29-476f-944a-956a3dcdb08e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:19.313332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ce59e-9f29-476f-944a-956a3dcdb08e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:19.313615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ce59e-9f29-476f-944a-956a3dcdb08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:19.363973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4a8ce59e-9f29-476f-944a-956a3dcdb08e None None] MFA Rules not processed for user `8bd57a7e4c074ca2890ec6bc081809cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:19.365196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a8ce59e-9f29-476f-944a-956a3dcdb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:19.398585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:19.437495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 699/1405] 10.222.0.58 () {58 vars in 1039 bytes} [Thu Jun 16 17:59:19 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:19.447436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89372a-f393-4253-9f00-6a13bb4d2c4d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:19.461029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89372a-f393-4253-9f00-6a13bb4d2c4d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:19.467099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796c494-1d77-416e-8c11-8bb92777946a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:19.467775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796c494-1d77-416e-8c11-8bb92777946a service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:19.468060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796c494-1d77-416e-8c11-8bb92777946a service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:19.468339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796c494-1d77-416e-8c11-8bb92777946a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:19.476449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796c494-1d77-416e-8c11-8bb92777946a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:19.478004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796c494-1d77-416e-8c11-8bb92777946a service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:19.487052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 707/1406] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 17:59:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:59:20.815160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8d1f-077d-4166-b971-cb1d252e8dc4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:20.815722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8d1f-077d-4166-b971-cb1d252e8dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:20.815842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8d1f-077d-4166-b971-cb1d252e8dc4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:20.817206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 700/1407] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 17:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:20.822365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206973-c5c2-4810-b568-ec72e04c37ca None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:20.822770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206973-c5c2-4810-b568-ec72e04c37ca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:20.823040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206973-c5c2-4810-b568-ec72e04c37ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:20.842048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f6206973-c5c2-4810-b568-ec72e04c37ca None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:20.843273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f6206973-c5c2-4810-b568-ec72e04c37ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:20.859222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 708/1408] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 17:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:20.898232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d8d1f-077d-4166-b971-cb1d252e8dc4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:20.918013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d8d1f-077d-4166-b971-cb1d252e8dc4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:20.923585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb277da-a786-441a-977a-0eac586b2e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:20.924237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb277da-a786-441a-977a-0eac586b2e44 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:20.924523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb277da-a786-441a-977a-0eac586b2e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:20.924797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb277da-a786-441a-977a-0eac586b2e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:20.933671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb277da-a786-441a-977a-0eac586b2e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:20.935251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb277da-a786-441a-977a-0eac586b2e44 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:20.946428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 701/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:30.835562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=55 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:30.941374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=21 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:59:43.855921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac227c-c456-4cc2-aa2e-5279eb2c403c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:43.856470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac227c-c456-4cc2-aa2e-5279eb2c403c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:43.856470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac227c-c456-4cc2-aa2e-5279eb2c403c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:43.857849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 709/1410] 10.222.0.58 () {54 vars in 918 bytes} [Thu Jun 16 17:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:43.863017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79862b06-ff2f-47e6-84d3-2a0f5b03f3fa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:43.863485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79862b06-ff2f-47e6-84d3-2a0f5b03f3fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:43.863624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79862b06-ff2f-47e6-84d3-2a0f5b03f3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:43.882023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-79862b06-ff2f-47e6-84d3-2a0f5b03f3fa None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:43.883226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-79862b06-ff2f-47e6-84d3-2a0f5b03f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:43.891861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 702/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jun 16 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:43.905848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f252451-2a7a-488d-b571-fc7cfc446342 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:43.906138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f252451-2a7a-488d-b571-fc7cfc446342 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:43.906415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f252451-2a7a-488d-b571-fc7cfc446342 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:43.907813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 710/1412] 10.222.0.58 () {54 vars in 918 bytes} [Thu Jun 16 17:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:43.912512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749bc9-d0d2-459e-b773-f6bb6a558b7a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:43.912799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749bc9-d0d2-459e-b773-f6bb6a558b7a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:43.913077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749bc9-d0d2-459e-b773-f6bb6a558b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:43.932628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a7749bc9-d0d2-459e-b773-f6bb6a558b7a None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:43.933844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a7749bc9-d0d2-459e-b773-f6bb6a558b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:43.943390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 703/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jun 16 17:59:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:46.545692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934d026-c0b4-44c6-a655-80835ab9a981 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:46.546226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934d026-c0b4-44c6-a655-80835ab9a981 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:46.546226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934d026-c0b4-44c6-a655-80835ab9a981 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:46.547571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:46.552767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5c7e3-9fc4-4e0f-bff0-5a27b8623b15 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:46.553157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5c7e3-9fc4-4e0f-bff0-5a27b8623b15 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:46.553321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5c7e3-9fc4-4e0f-bff0-5a27b8623b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:46.571136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8fa5c7e3-9fc4-4e0f-bff0-5a27b8623b15 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:46.573084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8fa5c7e3-9fc4-4e0f-bff0-5a27b8623b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:46.580830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:46.623848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934d026-c0b4-44c6-a655-80835ab9a981 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:46.638399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934d026-c0b4-44c6-a655-80835ab9a981 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:46.643909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dfda9-c045-421b-b7a5-682926a0f502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:46.644548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dfda9-c045-421b-b7a5-682926a0f502 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:46.644837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dfda9-c045-421b-b7a5-682926a0f502 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:46.645116 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dfda9-c045-421b-b7a5-682926a0f502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:46.653638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dfda9-c045-421b-b7a5-682926a0f502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:46.655200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dfda9-c045-421b-b7a5-682926a0f502 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:46.664637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 712/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:47.625413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e14e-35e0-4ba3-a89f-fa1a0b81da3e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:47.625982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e14e-35e0-4ba3-a89f-fa1a0b81da3e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:47.625982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789e14e-35e0-4ba3-a89f-fa1a0b81da3e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:47.627401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:47.632426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e69e6-3cf0-4713-9e46-5ee653332f69 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:47.632870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e69e6-3cf0-4713-9e46-5ee653332f69 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:47.632967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e69e6-3cf0-4713-9e46-5ee653332f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:47.652478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-973e69e6-3cf0-4713-9e46-5ee653332f69 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:47.653670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-973e69e6-3cf0-4713-9e46-5ee653332f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:47.661336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:47.813079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e14e-35e0-4ba3-a89f-fa1a0b81da3e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:47.828069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789e14e-35e0-4ba3-a89f-fa1a0b81da3e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:47.833547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4c9a-ff31-499e-b51a-7f97f091399f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:47.834223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4c9a-ff31-499e-b51a-7f97f091399f service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:47.834555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4c9a-ff31-499e-b51a-7f97f091399f service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:47.834838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4c9a-ff31-499e-b51a-7f97f091399f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:47.843192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c4c9a-ff31-499e-b51a-7f97f091399f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:47.844841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c4c9a-ff31-499e-b51a-7f97f091399f service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:47.854464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 706/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:48.797886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb03e2-9824-4841-b689-28284e304130 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:48.798472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb03e2-9824-4841-b689-28284e304130 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:48.798472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb03e2-9824-4841-b689-28284e304130 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:48.800682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:48.806493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaebc47-7c9e-4980-a208-39b28bb50015 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:48.806949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaebc47-7c9e-4980-a208-39b28bb50015 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:48.807054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaebc47-7c9e-4980-a208-39b28bb50015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:48.825574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-edaebc47-7c9e-4980-a208-39b28bb50015 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:48.826789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-edaebc47-7c9e-4980-a208-39b28bb50015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:48.834665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:48.987594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb03e2-9824-4841-b689-28284e304130 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:49.001834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb03e2-9824-4841-b689-28284e304130 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:49.007465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80048e-1867-49fe-b511-81d358f02a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:49.008106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80048e-1867-49fe-b511-81d358f02a8a service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:49.016051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80048e-1867-49fe-b511-81d358f02a8a service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:49.016051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80048e-1867-49fe-b511-81d358f02a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:49.017856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80048e-1867-49fe-b511-81d358f02a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:49.019448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80048e-1867-49fe-b511-81d358f02a8a service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:49.029588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 715/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:49.952196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deba7b-be5a-4c90-9382-a7e8efcf18d0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:49.952772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deba7b-be5a-4c90-9382-a7e8efcf18d0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:49.952772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deba7b-be5a-4c90-9382-a7e8efcf18d0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:49.954165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:49.959061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473363e4-dcb4-43ec-a76a-d99129a70c89 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:49.959535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473363e4-dcb4-43ec-a76a-d99129a70c89 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:49.959635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473363e4-dcb4-43ec-a76a-d99129a70c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:49.977332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-473363e4-dcb4-43ec-a76a-d99129a70c89 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:49.978537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-473363e4-dcb4-43ec-a76a-d99129a70c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:49.986281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:50.140333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deba7b-be5a-4c90-9382-a7e8efcf18d0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:50.154723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deba7b-be5a-4c90-9382-a7e8efcf18d0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:50.160075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01728595-6938-438e-8fb3-147ea9f7e8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:50.160702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01728595-6938-438e-8fb3-147ea9f7e8b1 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:50.160983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01728595-6938-438e-8fb3-147ea9f7e8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:50.161257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01728595-6938-438e-8fb3-147ea9f7e8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:50.169691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01728595-6938-438e-8fb3-147ea9f7e8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:50.171242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01728595-6938-438e-8fb3-147ea9f7e8b1 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:50.181240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 709/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:51.104556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462c67a-410d-445a-b387-35691aa6b0e2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:51.105107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462c67a-410d-445a-b387-35691aa6b0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:51.105107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462c67a-410d-445a-b387-35691aa6b0e2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:51.106481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:59:51.111563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d9661-675d-4978-989f-5a52937233db None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:51.111994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d9661-675d-4978-989f-5a52937233db None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:51.112125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d9661-675d-4978-989f-5a52937233db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:51.130095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-036d9661-675d-4978-989f-5a52937233db None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:51.131279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-036d9661-675d-4978-989f-5a52937233db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:51.139459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:51.289962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462c67a-410d-445a-b387-35691aa6b0e2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:51.304362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462c67a-410d-445a-b387-35691aa6b0e2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:51.310095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51db6f-ea8b-4e12-ac6e-e1d78f2a04ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:51.310656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51db6f-ea8b-4e12-ac6e-e1d78f2a04ea service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:51.310944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51db6f-ea8b-4e12-ac6e-e1d78f2a04ea service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:51.311220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51db6f-ea8b-4e12-ac6e-e1d78f2a04ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:51.319569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51db6f-ea8b-4e12-ac6e-e1d78f2a04ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:51.321838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51db6f-ea8b-4e12-ac6e-e1d78f2a04ea service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:51.331336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 718/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:52.244037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf151a-1a35-49b6-99bc-78df8eb6466e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:52.244582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf151a-1a35-49b6-99bc-78df8eb6466e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:52.244582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf151a-1a35-49b6-99bc-78df8eb6466e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:52.245896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:52.251004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652472d-1f70-4612-bc49-f886c9300141 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:52.251490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652472d-1f70-4612-bc49-f886c9300141 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:52.251637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652472d-1f70-4612-bc49-f886c9300141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:52.270373 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2652472d-1f70-4612-bc49-f886c9300141 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:52.271605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2652472d-1f70-4612-bc49-f886c9300141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:52.279487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:52.435476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf151a-1a35-49b6-99bc-78df8eb6466e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:52.450215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf151a-1a35-49b6-99bc-78df8eb6466e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:52.455867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea9515-16a5-4335-9d3d-c80222efbd48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:52.456506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9515-16a5-4335-9d3d-c80222efbd48 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:52.456789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9515-16a5-4335-9d3d-c80222efbd48 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:52.457065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea9515-16a5-4335-9d3d-c80222efbd48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:52.466122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea9515-16a5-4335-9d3d-c80222efbd48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:52.467724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea9515-16a5-4335-9d3d-c80222efbd48 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:52.477069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 712/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:53.380141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2dd30-2e7d-4521-ab4c-8e832ea5048a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:53.380784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2dd30-2e7d-4521-ab4c-8e832ea5048a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:53.380784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2dd30-2e7d-4521-ab4c-8e832ea5048a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:53.382137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:53.387725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff69390-4d5c-4183-aaef-166625b06665 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:53.388183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff69390-4d5c-4183-aaef-166625b06665 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:53.388292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff69390-4d5c-4183-aaef-166625b06665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:53.406662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0ff69390-4d5c-4183-aaef-166625b06665 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:53.407902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0ff69390-4d5c-4183-aaef-166625b06665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:53.415647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:53.568096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2dd30-2e7d-4521-ab4c-8e832ea5048a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:53.582876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2dd30-2e7d-4521-ab4c-8e832ea5048a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:53.589084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc80db-b45a-4c32-ad43-dd44b4fd4a25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:53.589725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc80db-b45a-4c32-ad43-dd44b4fd4a25 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:53.590010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc80db-b45a-4c32-ad43-dd44b4fd4a25 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:53.590287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc80db-b45a-4c32-ad43-dd44b4fd4a25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:53.598318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc80db-b45a-4c32-ad43-dd44b4fd4a25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:53.599913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc80db-b45a-4c32-ad43-dd44b4fd4a25 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:53.609300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 721/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:54.532914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a0627-b565-435a-9ac1-6e7ac1b9b250 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:54.532914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a0627-b565-435a-9ac1-6e7ac1b9b250 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:54.533407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a0627-b565-435a-9ac1-6e7ac1b9b250 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:54.534631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:54.540105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f71c1-7b17-4092-a93c-368e66251735 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:54.540582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f71c1-7b17-4092-a93c-368e66251735 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:54.540731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f71c1-7b17-4092-a93c-368e66251735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:54.559009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ae6f71c1-7b17-4092-a93c-368e66251735 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:54.560298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ae6f71c1-7b17-4092-a93c-368e66251735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:54.568241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:54.722614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a0627-b565-435a-9ac1-6e7ac1b9b250 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:54.737223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a0627-b565-435a-9ac1-6e7ac1b9b250 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:54.743631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7629b-6d7c-4595-87ee-e767a1b55340 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:54.744271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7629b-6d7c-4595-87ee-e767a1b55340 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:54.744551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7629b-6d7c-4595-87ee-e767a1b55340 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:54.744820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7629b-6d7c-4595-87ee-e767a1b55340 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:54.752795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7629b-6d7c-4595-87ee-e767a1b55340 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:54.754602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7629b-6d7c-4595-87ee-e767a1b55340 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:54.764788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 715/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:55.690517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d71265-d4dd-4c3e-a00f-14ee4e42fefc None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:55.691080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d71265-d4dd-4c3e-a00f-14ee4e42fefc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:55.691080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d71265-d4dd-4c3e-a00f-14ee4e42fefc None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:55.692493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:55.698150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9cb78-6782-4aa1-8093-0c7bbe204c96 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:55.698576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9cb78-6782-4aa1-8093-0c7bbe204c96 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:55.698731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9cb78-6782-4aa1-8093-0c7bbe204c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:55.717405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-eee9cb78-6782-4aa1-8093-0c7bbe204c96 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:55.718608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eee9cb78-6782-4aa1-8093-0c7bbe204c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:55.726263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:59:55.879646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d71265-d4dd-4c3e-a00f-14ee4e42fefc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:55.896620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d71265-d4dd-4c3e-a00f-14ee4e42fefc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:55.902864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b53546-d4e1-418e-835a-6aa48d897b93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:55.903554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b53546-d4e1-418e-835a-6aa48d897b93 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:55.903846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b53546-d4e1-418e-835a-6aa48d897b93 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:55.904128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b53546-d4e1-418e-835a-6aa48d897b93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:55.912535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b53546-d4e1-418e-835a-6aa48d897b93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:55.914154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b53546-d4e1-418e-835a-6aa48d897b93 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:55.923667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 724/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:59:57.955334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87554-4812-4cc3-9fd3-04d205e65191 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:57.955917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87554-4812-4cc3-9fd3-04d205e65191 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:57.955917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b87554-4812-4cc3-9fd3-04d205e65191 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:57.957312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:59:57.963054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8f948-6681-4eee-aef7-e2f59b1de59f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:57.963524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8f948-6681-4eee-aef7-e2f59b1de59f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:57.963663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8f948-6681-4eee-aef7-e2f59b1de59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:57.981063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ddc8f948-6681-4eee-aef7-e2f59b1de59f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:59:57.982278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ddc8f948-6681-4eee-aef7-e2f59b1de59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:59:57.990107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:59:58.029586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87554-4812-4cc3-9fd3-04d205e65191 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:59:58.040487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b87554-4812-4cc3-9fd3-04d205e65191 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:59:58.044028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9d68e-41f8-45f6-8d1a-7e973ade9f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:59:58.044664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9d68e-41f8-45f6-8d1a-7e973ade9f08 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:59:58.044952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9d68e-41f8-45f6-8d1a-7e973ade9f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:59:58.045228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9d68e-41f8-45f6-8d1a-7e973ade9f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:59:58.054047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9d68e-41f8-45f6-8d1a-7e973ade9f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:59:58.055716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9d68e-41f8-45f6-8d1a-7e973ade9f08 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:59:58.065402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 718/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 17:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:00.387894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c73b-737d-4700-990c-0d74dae0fe0d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:00.388500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c73b-737d-4700-990c-0d74dae0fe0d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:00.388500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33c73b-737d-4700-990c-0d74dae0fe0d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:00.389834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:00.395062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ac2e-2ff8-4305-9545-8dbcb8830c59 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:00.395557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ac2e-2ff8-4305-9545-8dbcb8830c59 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:00.395669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ac2e-2ff8-4305-9545-8dbcb8830c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:00.413042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-72a2ac2e-2ff8-4305-9545-8dbcb8830c59 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:00.414792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-72a2ac2e-2ff8-4305-9545-8dbcb8830c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:00.422622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:00.460342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33c73b-737d-4700-990c-0d74dae0fe0d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:00.472148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33c73b-737d-4700-990c-0d74dae0fe0d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:00.475590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60442b38-99af-493c-9cd0-8c484d02cee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:00.477070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442b38-99af-493c-9cd0-8c484d02cee8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:00.477655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442b38-99af-493c-9cd0-8c484d02cee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:00.477885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442b38-99af-493c-9cd0-8c484d02cee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:00.487260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60442b38-99af-493c-9cd0-8c484d02cee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:00.488725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60442b38-99af-493c-9cd0-8c484d02cee8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:00.498327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 727/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:03.048871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367be7-a64d-4426-854c-fa86bbab2d36 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:03.049445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367be7-a64d-4426-854c-fa86bbab2d36 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:03.049558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93367be7-a64d-4426-854c-fa86bbab2d36 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:03.050906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:03.056907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d122661-1ec0-47a0-b3be-bda382c343c8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:03.057326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d122661-1ec0-47a0-b3be-bda382c343c8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:03.057477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d122661-1ec0-47a0-b3be-bda382c343c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:03.075829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6d122661-1ec0-47a0-b3be-bda382c343c8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:03.077045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6d122661-1ec0-47a0-b3be-bda382c343c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:03.085122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:03.124264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367be7-a64d-4426-854c-fa86bbab2d36 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:03.135208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93367be7-a64d-4426-854c-fa86bbab2d36 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:03.138770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a4299-5aed-46d6-b729-9c465f77a84b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:03.139434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a4299-5aed-46d6-b729-9c465f77a84b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:03.139729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a4299-5aed-46d6-b729-9c465f77a84b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:03.140010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a4299-5aed-46d6-b729-9c465f77a84b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:03.148894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a4299-5aed-46d6-b729-9c465f77a84b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:03.150306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a4299-5aed-46d6-b729-9c465f77a84b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:03.159684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 721/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:05.431741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a8ab5-a0ce-4af6-ba85-588c71ef1bda None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:05.432291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a8ab5-a0ce-4af6-ba85-588c71ef1bda None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:05.432291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a8ab5-a0ce-4af6-ba85-588c71ef1bda None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:05.433667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:05.438824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c477c0-b755-40dd-9510-dd16af5a94eb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:05.439234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c477c0-b755-40dd-9510-dd16af5a94eb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:05.439417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c477c0-b755-40dd-9510-dd16af5a94eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:05.458545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b1c477c0-b755-40dd-9510-dd16af5a94eb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:05.459789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b1c477c0-b755-40dd-9510-dd16af5a94eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:05.468399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:05.506730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a8ab5-a0ce-4af6-ba85-588c71ef1bda None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:05.518183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a8ab5-a0ce-4af6-ba85-588c71ef1bda None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:05.521713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9e155-d803-469e-bf11-00954d9c308c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:05.522353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9e155-d803-469e-bf11-00954d9c308c service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:05.523209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9e155-d803-469e-bf11-00954d9c308c service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:05.523517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9e155-d803-469e-bf11-00954d9c308c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:05.532323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9e155-d803-469e-bf11-00954d9c308c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:05.533747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9e155-d803-469e-bf11-00954d9c308c service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:05.543024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 730/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:07.735150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d5f0-3522-4004-8658-f9d12cea2dc7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:07.735768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d5f0-3522-4004-8658-f9d12cea2dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:07.735768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d5f0-3522-4004-8658-f9d12cea2dc7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:07.737142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:07.742595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde8f8f-4bd1-4c66-bc3f-af469b18bb70 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:07.743024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde8f8f-4bd1-4c66-bc3f-af469b18bb70 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:07.743159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde8f8f-4bd1-4c66-bc3f-af469b18bb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:07.761214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1cde8f8f-4bd1-4c66-bc3f-af469b18bb70 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:07.762420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1cde8f8f-4bd1-4c66-bc3f-af469b18bb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:07.770265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:07.808197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d5f0-3522-4004-8658-f9d12cea2dc7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:07.818744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d5f0-3522-4004-8658-f9d12cea2dc7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:07.823161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19d8d0-776e-4134-9ee6-22d84ecd51ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:07.823941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19d8d0-776e-4134-9ee6-22d84ecd51ea service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:07.824223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19d8d0-776e-4134-9ee6-22d84ecd51ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:07.824504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19d8d0-776e-4134-9ee6-22d84ecd51ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:07.833398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19d8d0-776e-4134-9ee6-22d84ecd51ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:07.835499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19d8d0-776e-4134-9ee6-22d84ecd51ea service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:07.844640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 724/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:10.051619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f1c9d-6897-43e3-be09-c82893043888 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:10.052164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f1c9d-6897-43e3-be09-c82893043888 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:10.052164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f1c9d-6897-43e3-be09-c82893043888 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:10.054321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:10.059724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3b158-f181-4314-a2bf-9661238f95c4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:10.060169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3b158-f181-4314-a2bf-9661238f95c4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:10.060278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3b158-f181-4314-a2bf-9661238f95c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:10.077960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c1c3b158-f181-4314-a2bf-9661238f95c4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:10.085290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c1c3b158-f181-4314-a2bf-9661238f95c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:10.087963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:10.126642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f1c9d-6897-43e3-be09-c82893043888 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:10.138000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f1c9d-6897-43e3-be09-c82893043888 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:10.141539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0848f4-fa9c-4451-b8b4-d415002f50a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:10.142303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0848f4-fa9c-4451-b8b4-d415002f50a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:10.142595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0848f4-fa9c-4451-b8b4-d415002f50a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:10.142876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0848f4-fa9c-4451-b8b4-d415002f50a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:10.152073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0848f4-fa9c-4451-b8b4-d415002f50a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:10.153497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0848f4-fa9c-4451-b8b4-d415002f50a5 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:10.163571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 733/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:12.374577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dcd7d-82b1-48dd-b82a-831da407a43f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:12.375155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dcd7d-82b1-48dd-b82a-831da407a43f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:12.375155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dcd7d-82b1-48dd-b82a-831da407a43f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:12.376554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:12.382217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c634f-b106-4958-b681-5f806d235032 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:12.382654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c634f-b106-4958-b681-5f806d235032 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:12.382771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c634f-b106-4958-b681-5f806d235032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:12.388801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=70 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:00:12.401537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-233c634f-b106-4958-b681-5f806d235032 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:12.405023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-233c634f-b106-4958-b681-5f806d235032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:12.410670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:12.449132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dcd7d-82b1-48dd-b82a-831da407a43f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:12.450811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-814dcd7d-82b1-48dd-b82a-831da407a43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:12.458355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=73 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:00:12.464872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dcd7d-82b1-48dd-b82a-831da407a43f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:12.468503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdefeaa-e1c5-4b0c-974a-e569e64a739b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:12.470197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdefeaa-e1c5-4b0c-974a-e569e64a739b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:12.470197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdefeaa-e1c5-4b0c-974a-e569e64a739b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:12.470197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdefeaa-e1c5-4b0c-974a-e569e64a739b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:12.479444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdefeaa-e1c5-4b0c-974a-e569e64a739b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:12.480861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdefeaa-e1c5-4b0c-974a-e569e64a739b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:12.490248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 727/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:14.672032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2021e-c7ac-4581-86be-d9a781dd9559 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:14.672574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2021e-c7ac-4581-86be-d9a781dd9559 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:14.672574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2021e-c7ac-4581-86be-d9a781dd9559 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:14.673966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:14.679951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7299d-0989-4a6a-9c6f-98f62f75f605 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:14.680370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7299d-0989-4a6a-9c6f-98f62f75f605 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:14.683171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7299d-0989-4a6a-9c6f-98f62f75f605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:14.698427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-37f7299d-0989-4a6a-9c6f-98f62f75f605 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:14.699663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-37f7299d-0989-4a6a-9c6f-98f62f75f605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:14.707664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:14.744960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2021e-c7ac-4581-86be-d9a781dd9559 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:14.755867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2021e-c7ac-4581-86be-d9a781dd9559 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:14.760137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b177e-9df5-4047-a46c-1d46f1d79670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:14.760776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b177e-9df5-4047-a46c-1d46f1d79670 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:14.761062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b177e-9df5-4047-a46c-1d46f1d79670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:14.761338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b177e-9df5-4047-a46c-1d46f1d79670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:14.770055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b177e-9df5-4047-a46c-1d46f1d79670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:14.771569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b177e-9df5-4047-a46c-1d46f1d79670 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:14.781007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 736/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:16.958508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3b01-47c6-495b-acca-ab9ee55e47f7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:16.959688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3b01-47c6-495b-acca-ab9ee55e47f7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:16.959688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3b01-47c6-495b-acca-ab9ee55e47f7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:16.961089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:16.966455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616e21e-ef7e-46c8-a1ad-dc2fbb834ae0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:16.966869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616e21e-ef7e-46c8-a1ad-dc2fbb834ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:16.967002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616e21e-ef7e-46c8-a1ad-dc2fbb834ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:16.985310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a616e21e-ef7e-46c8-a1ad-dc2fbb834ae0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:16.986517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a616e21e-ef7e-46c8-a1ad-dc2fbb834ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:16.994423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:17.031839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3b01-47c6-495b-acca-ab9ee55e47f7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:17.043759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3b01-47c6-495b-acca-ab9ee55e47f7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:17.047242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005850eb-1662-4c3a-89a2-ab4efe31aa48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:17.048004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005850eb-1662-4c3a-89a2-ab4efe31aa48 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:17.055780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005850eb-1662-4c3a-89a2-ab4efe31aa48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:17.055780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005850eb-1662-4c3a-89a2-ab4efe31aa48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:17.057617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005850eb-1662-4c3a-89a2-ab4efe31aa48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:17.059027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005850eb-1662-4c3a-89a2-ab4efe31aa48 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:17.068844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 730/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:19.219680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04aa13-a37f-4e2c-a967-1e0712a8ecf1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:19.220224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04aa13-a37f-4e2c-a967-1e0712a8ecf1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:19.220224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04aa13-a37f-4e2c-a967-1e0712a8ecf1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:19.221614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:19.229351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08b8dd-e407-4766-803d-eb00375b35ba None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:19.229851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08b8dd-e407-4766-803d-eb00375b35ba None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:19.229949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08b8dd-e407-4766-803d-eb00375b35ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:19.248279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0f08b8dd-e407-4766-803d-eb00375b35ba None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:19.249328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f08b8dd-e407-4766-803d-eb00375b35ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:19.259023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:19.295328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04aa13-a37f-4e2c-a967-1e0712a8ecf1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:19.306724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04aa13-a37f-4e2c-a967-1e0712a8ecf1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:19.310213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae982e5-5566-4a7d-98d8-89df6ec79576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:19.310852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae982e5-5566-4a7d-98d8-89df6ec79576 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:19.311138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae982e5-5566-4a7d-98d8-89df6ec79576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:19.311456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae982e5-5566-4a7d-98d8-89df6ec79576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:19.320860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae982e5-5566-4a7d-98d8-89df6ec79576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:19.322282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae982e5-5566-4a7d-98d8-89df6ec79576 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:19.331641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 739/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:29.242871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=17 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:00:29.327506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=17 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:00:40.343261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e75d-7718-49b8-928d-da90d14e0acb None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:40.343888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e75d-7718-49b8-928d-da90d14e0acb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:40.343888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23e75d-7718-49b8-928d-da90d14e0acb None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:40.345322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 732/1471] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:40.351145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb17f82-0a21-4c57-8eaf-b580a894fa1e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:40.351664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb17f82-0a21-4c57-8eaf-b580a894fa1e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:40.351770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb17f82-0a21-4c57-8eaf-b580a894fa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:40.369784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3cb17f82-0a21-4c57-8eaf-b580a894fa1e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:40.371016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3cb17f82-0a21-4c57-8eaf-b580a894fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:40.381672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 740/1472] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:40.438194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd10f-bbee-4006-8c58-071f455683d8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:40.438483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd10f-bbee-4006-8c58-071f455683d8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:40.438758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bd10f-bbee-4006-8c58-071f455683d8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:40.440161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 733/1473] 10.222.0.58 () {54 vars in 962 bytes} [Thu Jun 16 18:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:40.444906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599831-d25c-4d64-9d8f-7a3c75544a2f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:40.445199 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599831-d25c-4d64-9d8f-7a3c75544a2f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:40.445484 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8599831-d25c-4d64-9d8f-7a3c75544a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:40.463812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b8599831-d25c-4d64-9d8f-7a3c75544a2f None None] MFA Rules not processed for user `6f37017740874fef845f23fca4f977f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:40.465012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b8599831-d25c-4d64-9d8f-7a3c75544a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:40.471460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 741/1474] 10.222.0.58 () {58 vars in 1059 bytes} [Thu Jun 16 18:00:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:40.479413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bd10f-bbee-4006-8c58-071f455683d8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:40.490425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bd10f-bbee-4006-8c58-071f455683d8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:40.493935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154e0a3-64ee-4ccb-85dd-52fe768d252d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:40.494560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e0a3-64ee-4ccb-85dd-52fe768d252d service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:40.494843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e0a3-64ee-4ccb-85dd-52fe768d252d service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:40.495121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e0a3-64ee-4ccb-85dd-52fe768d252d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:40.512763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154e0a3-64ee-4ccb-85dd-52fe768d252d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:40.514162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154e0a3-64ee-4ccb-85dd-52fe768d252d service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:40.524288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 734/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:42.758595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264bffc-38b9-412a-98b8-ed71011231f9 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:42.759226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264bffc-38b9-412a-98b8-ed71011231f9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:42.759226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264bffc-38b9-412a-98b8-ed71011231f9 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:42.760602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 742/1476] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:42.765654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecf146-049c-4c2f-b162-9f154382a3b9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:42.766098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecf146-049c-4c2f-b162-9f154382a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:42.766217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecf146-049c-4c2f-b162-9f154382a3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:42.785672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f3ecf146-049c-4c2f-b162-9f154382a3b9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:42.786864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f3ecf146-049c-4c2f-b162-9f154382a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:42.794543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 735/1477] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:42.817927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264bffc-38b9-412a-98b8-ed71011231f9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:42.830105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264bffc-38b9-412a-98b8-ed71011231f9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:42.835322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:42.835991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:42.836272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:42.836545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:42.837278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:42.839028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:00:42.839028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:00:42.839028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:00:42.840195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d0473-553a-4b8e-ac7d-2cdce803ee1a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:42.859043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 743/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Thu Jun 16 18:00:42 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:00:45.056918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054900d-8f32-448a-8308-65e6ab0c354a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:45.057491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054900d-8f32-448a-8308-65e6ab0c354a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:45.057491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054900d-8f32-448a-8308-65e6ab0c354a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:45.058998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 736/1479] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:45.064656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919330-f73f-4967-aaeb-ec160a6c9c2c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:45.065464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919330-f73f-4967-aaeb-ec160a6c9c2c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:45.065464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1919330-f73f-4967-aaeb-ec160a6c9c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:45.085404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e1919330-f73f-4967-aaeb-ec160a6c9c2c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:45.086686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e1919330-f73f-4967-aaeb-ec160a6c9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:45.094572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 744/1480] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:45.139147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054900d-8f32-448a-8308-65e6ab0c354a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:45.153594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054900d-8f32-448a-8308-65e6ab0c354a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:45.159042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed306c3b-9596-4226-a126-e5d7185d6879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:45.160370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed306c3b-9596-4226-a126-e5d7185d6879 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:45.160608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed306c3b-9596-4226-a126-e5d7185d6879 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:45.160887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed306c3b-9596-4226-a126-e5d7185d6879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:45.169018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed306c3b-9596-4226-a126-e5d7185d6879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:45.170585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed306c3b-9596-4226-a126-e5d7185d6879 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:45.180550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 737/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:47.539485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26135-d585-4bdd-8631-0a9487d22f10 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:47.540028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26135-d585-4bdd-8631-0a9487d22f10 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:47.540028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26135-d585-4bdd-8631-0a9487d22f10 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:47.541410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 745/1482] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:47.546449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643273-7754-44f9-9ffc-e686453af87d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:47.546849 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643273-7754-44f9-9ffc-e686453af87d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:47.547005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24643273-7754-44f9-9ffc-e686453af87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:47.564912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-24643273-7754-44f9-9ffc-e686453af87d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:47.566665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-24643273-7754-44f9-9ffc-e686453af87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:47.574467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 738/1483] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:47.610738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26135-d585-4bdd-8631-0a9487d22f10 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:47.621646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26135-d585-4bdd-8631-0a9487d22f10 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:47.625159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e0734-1559-4e81-a483-377042e6b565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:47.625795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e0734-1559-4e81-a483-377042e6b565 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:47.626080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e0734-1559-4e81-a483-377042e6b565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:47.626356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e0734-1559-4e81-a483-377042e6b565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:47.635990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e0734-1559-4e81-a483-377042e6b565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:47.637413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e0734-1559-4e81-a483-377042e6b565 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:47.647619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 746/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:49.844427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857da7-5af8-452f-a48c-369870483f4f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:49.844968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857da7-5af8-452f-a48c-369870483f4f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:49.844968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857da7-5af8-452f-a48c-369870483f4f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:49.847058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 739/1485] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:49.852457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1132f-e75b-4b8a-b9ce-acbca3948c7a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:49.852899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1132f-e75b-4b8a-b9ce-acbca3948c7a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:49.853006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1132f-e75b-4b8a-b9ce-acbca3948c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:49.870936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d4c1132f-e75b-4b8a-b9ce-acbca3948c7a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:49.872173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4c1132f-e75b-4b8a-b9ce-acbca3948c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:49.880500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 747/1486] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:49.917984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857da7-5af8-452f-a48c-369870483f4f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:49.929479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857da7-5af8-452f-a48c-369870483f4f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:49.932993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b3653-9685-47e4-8b9c-dfafdda1fb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:49.933627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b3653-9685-47e4-8b9c-dfafdda1fb44 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:49.933919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b3653-9685-47e4-8b9c-dfafdda1fb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:49.934198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b3653-9685-47e4-8b9c-dfafdda1fb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:49.943212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b3653-9685-47e4-8b9c-dfafdda1fb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:49.944682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b3653-9685-47e4-8b9c-dfafdda1fb44 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:49.954122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 740/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:52.245553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43eb2-88b5-47d3-8c9d-230435407a32 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:52.246079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43eb2-88b5-47d3-8c9d-230435407a32 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:52.246079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43eb2-88b5-47d3-8c9d-230435407a32 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:52.247921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 748/1488] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:52.253814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab674fb-803d-4637-afd1-a6dab83ab52b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:52.254705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab674fb-803d-4637-afd1-a6dab83ab52b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:52.254705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab674fb-803d-4637-afd1-a6dab83ab52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:52.272383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4ab674fb-803d-4637-afd1-a6dab83ab52b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:52.273584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ab674fb-803d-4637-afd1-a6dab83ab52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:52.281318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 741/1489] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:52.323193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43eb2-88b5-47d3-8c9d-230435407a32 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:52.334396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43eb2-88b5-47d3-8c9d-230435407a32 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:52.337837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93d700-32c1-42d9-821a-eea5c53530d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:52.338469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d700-32c1-42d9-821a-eea5c53530d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:52.338747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d700-32c1-42d9-821a-eea5c53530d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:52.339020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d700-32c1-42d9-821a-eea5c53530d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:52.348217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93d700-32c1-42d9-821a-eea5c53530d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:52.349619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93d700-32c1-42d9-821a-eea5c53530d2 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:52.358769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 749/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:00:54.906836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b6e6-4653-4e66-ab95-7cec33be9424 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:54.907359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b6e6-4653-4e66-ab95-7cec33be9424 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:54.907359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5b6e6-4653-4e66-ab95-7cec33be9424 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:54.909308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 742/1491] 10.222.0.58 () {54 vars in 926 bytes} [Thu Jun 16 18:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:00:54.914785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf47b63-1fa4-4030-b749-b3b3ff1af778 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:54.915198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf47b63-1fa4-4030-b749-b3b3ff1af778 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:54.915350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf47b63-1fa4-4030-b749-b3b3ff1af778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:54.933431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5cf47b63-1fa4-4030-b749-b3b3ff1af778 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:00:54.934635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5cf47b63-1fa4-4030-b749-b3b3ff1af778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:00:54.942454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 750/1492] 10.222.0.58 () {58 vars in 1023 bytes} [Thu Jun 16 18:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:00:54.981950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5b6e6-4653-4e66-ab95-7cec33be9424 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:00:54.993361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5b6e6-4653-4e66-ab95-7cec33be9424 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:00:54.996847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30f70b-8897-4f78-bd20-de5e5d8e49b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:00:54.997479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30f70b-8897-4f78-bd20-de5e5d8e49b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:00:54.997789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30f70b-8897-4f78-bd20-de5e5d8e49b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:00:54.998065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30f70b-8897-4f78-bd20-de5e5d8e49b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:00:55.007161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30f70b-8897-4f78-bd20-de5e5d8e49b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:00:55.008643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30f70b-8897-4f78-bd20-de5e5d8e49b6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:00:55.017910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 743/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:01:04.928364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=33 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:01:05.013738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=33 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:02:30.327349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.327939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.327939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.345901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:02:30.347107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:30.355048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 751/1494] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:02:30.364254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.364769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.364879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.383117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:02:30.384427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:30.393026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 744/1495] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:02:30.397417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:30.407502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16093ed-d51e-4786-a89c-ae3e6b0ecd50 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:30.411906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:30.412550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.412835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.413107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.413818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:02:30.414119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:30.415119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:30.415119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:02:30.415119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:02:30.417019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:30.428276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 752/1496] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:02:30.436734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:30.445867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4f888-e845-4213-b1d4-647a4632886e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:30.450570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:30.451217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.451532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.451812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.452512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:30.454275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:30.454275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:02:30.454275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:02:30.455478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:30.590768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 745/1497] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:02:30 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:02:30.594863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:30.605810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ca9f9-a3b4-41a7-a3a9-2fd8d8341db7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:30.661230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:30.661897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.662181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.662468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.663160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:30.664205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:30.664205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:02:30.664205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:02:30.665423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:30.764258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 753/1498] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:02:30 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:02:30.768417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:30.779330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb854bc-e2db-478d-8e1d-56c55967a2f7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:30.784022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:30.784685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.785015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.785302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.786061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:30.787068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:30.787068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:02:30.787068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:02:30.788017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:30.801735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 746/1499] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:02:30.806077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:30.816039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7d35-676a-471a-97cc-854a2751195f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:30.821037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:30.821772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.822369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.822662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.823417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:30.824438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:30.824438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:02:30.824438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:02:30.825329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970339b-f2cd-4e1f-a914-12ab86a58bd9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:30.839421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 754/1500] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:02:30.843565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:30.853426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c677eb2-e918-4a79-a852-aa3c597940da admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:30.858485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:30.859358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.859689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.859974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] PATH_INFO: `/v3/projects/0e223a1aa1d74599adbcc8c5bfe9d204/users/a1ac450989b14274a4828bc910e4c67f/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:30.875092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e223a1aa1d74599adbcc8c5bfe9d204, user_id=a1ac450989b14274a4828bc910e4c67f, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:30.876154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:30.876154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:02:30.876154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:02:30.878084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7dece-80a2-48f5-9de5-6534f43f833c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:30.960217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 747/1501] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:02:30 2022] PUT /identity/v3/projects/0e223a1aa1d74599adbcc8c5bfe9d204/users/a1ac450989b14274a4828bc910e4c67f/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:02:30.976090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45618b-e90b-4ed8-8c93-02292f9297fd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:30.976386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45618b-e90b-4ed8-8c93-02292f9297fd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:30.976669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45618b-e90b-4ed8-8c93-02292f9297fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.003729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4e45618b-e90b-4ed8-8c93-02292f9297fd None None] MFA Rules not processed for user `a1ac450989b14274a4828bc910e4c67f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:02:31.004959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4e45618b-e90b-4ed8-8c93-02292f9297fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:31.077019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 755/1502] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:02:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:02:31.091218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ff2d-4f48-45c3-b484-9ffc7da47494 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.091542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ff2d-4f48-45c3-b484-9ffc7da47494 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.091823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5ff2d-4f48-45c3-b484-9ffc7da47494 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.093185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 748/1503] 10.222.0.58 () {54 vars in 963 bytes} [Thu Jun 16 18:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:02:31.098041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9746583-f56b-415c-a70c-4fb7dfa999e8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.098276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9746583-f56b-415c-a70c-4fb7dfa999e8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.098556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9746583-f56b-415c-a70c-4fb7dfa999e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.117088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a9746583-f56b-415c-a70c-4fb7dfa999e8 None None] MFA Rules not processed for user `fbc41720695b45ec84c2f250238ac37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:02:31.118290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a9746583-f56b-415c-a70c-4fb7dfa999e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:31.162272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:02:31.169708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 756/1504] 10.222.0.58 () {58 vars in 1060 bytes} [Thu Jun 16 18:02:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:02:31.177458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5ff2d-4f48-45c3-b484-9ffc7da47494 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:31.188155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5ff2d-4f48-45c3-b484-9ffc7da47494 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:31.191636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:31.192264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.192546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.192817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.201595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:31.202999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:31.211688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 749/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:02:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:02:31.254866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c10e00-f709-425b-be87-5724d0ff0cc3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.255159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c10e00-f709-425b-be87-5724d0ff0cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.255465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c10e00-f709-425b-be87-5724d0ff0cc3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.256788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 757/1506] 10.222.0.58 () {52 vars in 829 bytes} [Thu Jun 16 18:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:02:31.260312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:31.321619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a04b93-6c1c-4568-b697-e1233cd5f097 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:31.326348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:31.327202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.327519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.327795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] PATH_INFO: `/v3/users/a1ac450989b14274a4828bc910e4c67f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.328953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ac450989b14274a4828bc910e4c67f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:31.329948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:31.329948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:02:31.329948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:02:31.331124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:31.532735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ac450989b14274a4828bc910e4c67f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:31.551992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ac450989b14274a4828bc910e4c67f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:31.564280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ac450989b14274a4828bc910e4c67f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:31.568266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:02:31.643842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ac450989b14274a4828bc910e4c67f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:31.644461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ac450989b14274a4828bc910e4c67f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:31.651558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 750/1507] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:02:31 2022] DELETE /identity/v3/users/a1ac450989b14274a4828bc910e4c67f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:02:31.656185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.656666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.656781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.674513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:02:31.675730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:31.734825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 758/1508] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:02:31.745415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:31.747109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:31.804870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1c4f6-d45f-4d20-a2d8-f01e71e0ef47 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:31.808675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27e070-59e2-47b7-a8c2-2997c4121c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:31.809325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e070-59e2-47b7-a8c2-2997c4121c29 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:31.809613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e070-59e2-47b7-a8c2-2997c4121c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:31.809893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e070-59e2-47b7-a8c2-2997c4121c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:31.818727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27e070-59e2-47b7-a8c2-2997c4121c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:31.820188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27e070-59e2-47b7-a8c2-2997c4121c29 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:31.829529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 751/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:02:32.320054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:02:32.321469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:02:32.334456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250ba46-e00a-4ace-89b4-964fa2016bdd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:02:32.339530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:02:32.340416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:02:32.340693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:02:32.340968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] PATH_INFO: `/v3/projects/0e223a1aa1d74599adbcc8c5bfe9d204` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:02:32.342198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e223a1aa1d74599adbcc8c5bfe9d204)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:02:32.344072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:02:32.344072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:02:32.344072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:02:32.344514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:02:32.477537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e223a1aa1d74599adbcc8c5bfe9d204', 'request_id': 'req-b8f0768d-28bc-4bb2-9c09-f069b3124475'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:32.484114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e223a1aa1d74599adbcc8c5bfe9d204', 'request_id': 'req-b8f0768d-28bc-4bb2-9c09-f069b3124475'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:32.499629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e223a1aa1d74599adbcc8c5bfe9d204', 'request_id': 'req-b8f0768d-28bc-4bb2-9c09-f069b3124475'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:32.795216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] The token cache is being invalidate because project 0e223a1aa1d74599adbcc8c5bfe9d204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:02:32.796036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:32.796501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8f0768d-28bc-4bb2-9c09-f069b3124475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:02:32.798055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 759/1510] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:02:32 2022] DELETE /identity/v3/projects/0e223a1aa1d74599adbcc8c5bfe9d204 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:02:41.825440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:02:42.791015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:04:16.197762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7e89f-7b29-4d9e-b935-a03ede5de69a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:16.198378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7e89f-7b29-4d9e-b935-a03ede5de69a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:16.198378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7e89f-7b29-4d9e-b935-a03ede5de69a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:16.199795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:16.204909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab6dc3-4553-4562-9601-cbea3d3a94ae None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:16.205377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab6dc3-4553-4562-9601-cbea3d3a94ae None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:16.205490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab6dc3-4553-4562-9601-cbea3d3a94ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:16.225087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1bab6dc3-4553-4562-9601-cbea3d3a94ae None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:16.226318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1bab6dc3-4553-4562-9601-cbea3d3a94ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:16.286902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:16.458322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7e89f-7b29-4d9e-b935-a03ede5de69a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:16.460053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e5b7e89f-7b29-4d9e-b935-a03ede5de69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:16.539484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7e89f-7b29-4d9e-b935-a03ede5de69a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:16.545464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd61c29-571b-471e-b498-a0fdebdd1f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:16.546126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd61c29-571b-471e-b498-a0fdebdd1f2f service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:16.546412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd61c29-571b-471e-b498-a0fdebdd1f2f service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:16.546693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd61c29-571b-471e-b498-a0fdebdd1f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:16.556720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd61c29-571b-471e-b498-a0fdebdd1f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:16.558368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd61c29-571b-471e-b498-a0fdebdd1f2f service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:16.569119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 753/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:17.490366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c9ce-de1a-4035-b939-0494ee46ee30 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:17.490884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c9ce-de1a-4035-b939-0494ee46ee30 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:17.490884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509c9ce-de1a-4035-b939-0494ee46ee30 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:17.492567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:17.497804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659b79d-a1ea-4e3e-a221-e04acf173b8b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:17.498779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659b79d-a1ea-4e3e-a221-e04acf173b8b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:17.498779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659b79d-a1ea-4e3e-a221-e04acf173b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:17.517350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8659b79d-a1ea-4e3e-a221-e04acf173b8b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:17.518575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8659b79d-a1ea-4e3e-a221-e04acf173b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:17.527186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:17.686319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509c9ce-de1a-4035-b939-0494ee46ee30 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:17.705172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509c9ce-de1a-4035-b939-0494ee46ee30 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:17.711009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf86630-54f3-4768-b1e2-3d21416ec44b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:17.711716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86630-54f3-4768-b1e2-3d21416ec44b service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:17.712805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86630-54f3-4768-b1e2-3d21416ec44b service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:17.712926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86630-54f3-4768-b1e2-3d21416ec44b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:17.722248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf86630-54f3-4768-b1e2-3d21416ec44b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:17.724053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf86630-54f3-4768-b1e2-3d21416ec44b service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:17.734157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 762/1516] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:17.750175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d6d7-c672-48b9-92e3-6a50576d4592 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:17.750581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d6d7-c672-48b9-92e3-6a50576d4592 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:17.750937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d6d7-c672-48b9-92e3-6a50576d4592 None None] PATH_INFO: `/v3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:17.752551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 755/1517] 10.222.0.18 () {54 vars in 925 bytes} [Thu Jun 16 18:04:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:04:17.762126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10df06-c3b3-46ac-96b4-a12dcb23e2d7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:17.762620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10df06-c3b3-46ac-96b4-a12dcb23e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:17.763104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10df06-c3b3-46ac-96b4-a12dcb23e2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:17.794216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1b10df06-c3b3-46ac-96b4-a12dcb23e2d7 None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:17.796376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b10df06-c3b3-46ac-96b4-a12dcb23e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:17.885842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 763/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jun 16 18:04:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:27.512020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=37 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:04:27.624917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903d0ce-da05-4559-af6d-6a0ff6bd09d7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:27.625213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903d0ce-da05-4559-af6d-6a0ff6bd09d7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:27.625492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903d0ce-da05-4559-af6d-6a0ff6bd09d7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:27.626902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 756/1519] 10.222.0.58 () {54 vars in 960 bytes} [Thu Jun 16 18:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:27.632049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9a199-0bdf-4c38-8e8b-b2a509679687 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:27.632525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9a199-0bdf-4c38-8e8b-b2a509679687 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:27.632651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9a199-0bdf-4c38-8e8b-b2a509679687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:27.652857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-98d9a199-0bdf-4c38-8e8b-b2a509679687 None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:27.654093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-98d9a199-0bdf-4c38-8e8b-b2a509679687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:27.662603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 764/1520] 10.222.0.58 () {58 vars in 1057 bytes} [Thu Jun 16 18:04:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:27.670783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903d0ce-da05-4559-af6d-6a0ff6bd09d7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:27.684232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903d0ce-da05-4559-af6d-6a0ff6bd09d7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:27.689960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919b75-c19f-4cad-88e2-30a7696d0c71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:27.690890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919b75-c19f-4cad-88e2-30a7696d0c71 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:27.690890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919b75-c19f-4cad-88e2-30a7696d0c71 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:27.691155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919b75-c19f-4cad-88e2-30a7696d0c71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:27.700506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7919b75-c19f-4cad-88e2-30a7696d0c71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:27.702088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7919b75-c19f-4cad-88e2-30a7696d0c71 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:27.712572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 757/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:29.047117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861776d-90f7-4f2f-a0ec-46697a708520 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:29.047723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861776d-90f7-4f2f-a0ec-46697a708520 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:29.047723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861776d-90f7-4f2f-a0ec-46697a708520 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:29.049088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:29.054203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f59de-0eca-4e36-b86b-3301629711d3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:29.056857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f59de-0eca-4e36-b86b-3301629711d3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:29.056857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f59de-0eca-4e36-b86b-3301629711d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:29.072882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-714f59de-0eca-4e36-b86b-3301629711d3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:29.074152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-714f59de-0eca-4e36-b86b-3301629711d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:29.082039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:29.239296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861776d-90f7-4f2f-a0ec-46697a708520 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:29.256893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861776d-90f7-4f2f-a0ec-46697a708520 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:29.263605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f13d1c-6abe-4927-a192-0cf073801db0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:29.264256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13d1c-6abe-4927-a192-0cf073801db0 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:29.264538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13d1c-6abe-4927-a192-0cf073801db0 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:29.264818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f13d1c-6abe-4927-a192-0cf073801db0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:29.275194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13d1c-6abe-4927-a192-0cf073801db0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:29.276819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f13d1c-6abe-4927-a192-0cf073801db0 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:29.286245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 766/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:30.170859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8510b87-1026-49b1-bda6-23a762f80502 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:30.171468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8510b87-1026-49b1-bda6-23a762f80502 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:30.171468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8510b87-1026-49b1-bda6-23a762f80502 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:30.172854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:30.177745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb09c7-db77-456b-8dc6-058c99fe284d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:30.178134 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb09c7-db77-456b-8dc6-058c99fe284d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:30.178300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb09c7-db77-456b-8dc6-058c99fe284d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:30.197583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dfeb09c7-db77-456b-8dc6-058c99fe284d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:30.198789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dfeb09c7-db77-456b-8dc6-058c99fe284d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:30.206480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:30.358941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8510b87-1026-49b1-bda6-23a762f80502 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:30.374138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8510b87-1026-49b1-bda6-23a762f80502 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:30.379553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98ecc0-dc7c-4663-b20f-e2df761326d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:30.380192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ecc0-dc7c-4663-b20f-e2df761326d8 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:30.380477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ecc0-dc7c-4663-b20f-e2df761326d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:30.380757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98ecc0-dc7c-4663-b20f-e2df761326d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:30.389970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98ecc0-dc7c-4663-b20f-e2df761326d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:30.391582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98ecc0-dc7c-4663-b20f-e2df761326d8 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:30.401169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 760/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:34.749271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629768c-0373-4ddf-ab41-9bf25d015a98 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:34.749824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629768c-0373-4ddf-ab41-9bf25d015a98 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:34.750018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629768c-0373-4ddf-ab41-9bf25d015a98 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:34.751473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:34.756645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b48b53-c653-43bb-ab49-4440bff77fe2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:34.757082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b48b53-c653-43bb-ab49-4440bff77fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:34.757195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b48b53-c653-43bb-ab49-4440bff77fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:34.775440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e2b48b53-c653-43bb-ab49-4440bff77fe2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:34.776679 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e2b48b53-c653-43bb-ab49-4440bff77fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:34.784719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:34.821467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629768c-0373-4ddf-ab41-9bf25d015a98 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:34.823270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8629768c-0373-4ddf-ab41-9bf25d015a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:34.879323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629768c-0373-4ddf-ab41-9bf25d015a98 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:34.882873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3170443-b0e6-48bd-a1bd-794d729d35d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:34.883542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3170443-b0e6-48bd-a1bd-794d729d35d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:34.883829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3170443-b0e6-48bd-a1bd-794d729d35d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:34.884105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3170443-b0e6-48bd-a1bd-794d729d35d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:34.893089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3170443-b0e6-48bd-a1bd-794d729d35d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:34.894250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3170443-b0e6-48bd-a1bd-794d729d35d5 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:34.899424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:04:34.904346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 769/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:37.072923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3f10c-7036-47c4-b9f8-319570694963 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:37.073540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3f10c-7036-47c4-b9f8-319570694963 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:37.073540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3f10c-7036-47c4-b9f8-319570694963 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:37.074878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:37.080167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57450f4a-b49e-4a93-96a2-659de6e2d0ee None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:37.080719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57450f4a-b49e-4a93-96a2-659de6e2d0ee None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:37.080832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57450f4a-b49e-4a93-96a2-659de6e2d0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:37.099141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-57450f4a-b49e-4a93-96a2-659de6e2d0ee None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:37.100377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-57450f4a-b49e-4a93-96a2-659de6e2d0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:37.110531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:04:37.149376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3f10c-7036-47c4-b9f8-319570694963 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:37.151097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fca3f10c-7036-47c4-b9f8-319570694963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:37.164884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca3f10c-7036-47c4-b9f8-319570694963 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:37.168491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de77880-a372-4ce5-a453-da96816dc374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:37.169136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77880-a372-4ce5-a453-da96816dc374 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:37.169424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77880-a372-4ce5-a453-da96816dc374 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:37.169704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77880-a372-4ce5-a453-da96816dc374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:37.178996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de77880-a372-4ce5-a453-da96816dc374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:37.180469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de77880-a372-4ce5-a453-da96816dc374 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:37.189967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 763/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:40.881095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9250759-d5d1-4eab-a17d-9e923b070a5c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:40.881694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9250759-d5d1-4eab-a17d-9e923b070a5c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:40.881694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9250759-d5d1-4eab-a17d-9e923b070a5c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:40.883041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:04:40.889018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63929d1e-b8e0-415a-a765-f9732131f517 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:40.889520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63929d1e-b8e0-415a-a765-f9732131f517 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:40.889618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63929d1e-b8e0-415a-a765-f9732131f517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:40.908183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-63929d1e-b8e0-415a-a765-f9732131f517 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:40.909412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-63929d1e-b8e0-415a-a765-f9732131f517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:40.917207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:41.186575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9250759-d5d1-4eab-a17d-9e923b070a5c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:41.188324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9250759-d5d1-4eab-a17d-9e923b070a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:41.248855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9250759-d5d1-4eab-a17d-9e923b070a5c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:41.251997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff4e89-0706-478a-b7a6-8158e27fa97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:41.252651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4e89-0706-478a-b7a6-8158e27fa97d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:41.252944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4e89-0706-478a-b7a6-8158e27fa97d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:41.253222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff4e89-0706-478a-b7a6-8158e27fa97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:41.262938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff4e89-0706-478a-b7a6-8158e27fa97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:41.264389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff4e89-0706-478a-b7a6-8158e27fa97d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:41.273684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 772/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:43.418761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17cdce-f4bf-4f26-9776-56313153b469 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:43.419302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17cdce-f4bf-4f26-9776-56313153b469 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:43.419523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17cdce-f4bf-4f26-9776-56313153b469 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:43.420908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:04:43.426456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba420-d36d-4ede-9224-2b4245c1c5f7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:43.427010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba420-d36d-4ede-9224-2b4245c1c5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:43.427010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ba420-d36d-4ede-9224-2b4245c1c5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:43.444967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f04ba420-d36d-4ede-9224-2b4245c1c5f7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:04:43.446158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f04ba420-d36d-4ede-9224-2b4245c1c5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:43.454287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:04:43.726313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17cdce-f4bf-4f26-9776-56313153b469 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:04:43.728098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f17cdce-f4bf-4f26-9776-56313153b469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:04:43.742104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17cdce-f4bf-4f26-9776-56313153b469 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:04:43.745596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99d88c-8272-4f77-b42d-c5b6cb6e0546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:04:43.746464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99d88c-8272-4f77-b42d-c5b6cb6e0546 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:04:43.746750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99d88c-8272-4f77-b42d-c5b6cb6e0546 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:04:43.747032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99d88c-8272-4f77-b42d-c5b6cb6e0546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:04:43.756157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99d88c-8272-4f77-b42d-c5b6cb6e0546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:04:43.757612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99d88c-8272-4f77-b42d-c5b6cb6e0546 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:04:43.768242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 766/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:04:53.439838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=32 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:04:53.763993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=33 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:17.961959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33492b-226d-4103-851d-94bfffdb0b7a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:17.963082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33492b-226d-4103-851d-94bfffdb0b7a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:17.963593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33492b-226d-4103-851d-94bfffdb0b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:17.987811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d33492b-226d-4103-851d-94bfffdb0b7a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:17.989097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d33492b-226d-4103-851d-94bfffdb0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.002981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 774/1540] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.014157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.015328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.016004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.040018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3d555-e20e-4d2e-9dcf-50d111e03f0c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.040590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3d555-e20e-4d2e-9dcf-50d111e03f0c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.041068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3d555-e20e-4d2e-9dcf-50d111e03f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.060923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.063314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.074804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2df3d555-e20e-4d2e-9dcf-50d111e03f0c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.084440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2df3d555-e20e-4d2e-9dcf-50d111e03f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.086537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 767/1541] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.091462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.117302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106acfcb-fe5e-4e6b-bcbd-a0ef7fc5487d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.117705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 775/1542] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.126200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.127272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.128173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.128909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.130145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:18.130802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.132211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:06:18.132211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:06:18.132211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:06:18.133552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.161114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 768/1543] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:18.166623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdc5b3-457f-403b-885e-a6032cd5519d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.167336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdc5b3-457f-403b-885e-a6032cd5519d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.168048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdc5b3-457f-403b-885e-a6032cd5519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.181810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.206601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ec7f-e212-413d-bc1a-7f8959b0a3aa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.208452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b6fdc5b3-457f-403b-885e-a6032cd5519d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.210963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b6fdc5b3-457f-403b-885e-a6032cd5519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.219404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff7f42-efde-4381-a586-a62e34de685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.221929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff7f42-efde-4381-a586-a62e34de685c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.223046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff7f42-efde-4381-a586-a62e34de685c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.224272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff7f42-efde-4381-a586-a62e34de685c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.226272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff7f42-efde-4381-a586-a62e34de685c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.229931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff7f42-efde-4381-a586-a62e34de685c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.231835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 776/1544] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.245295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.246079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.246573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.309321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.311537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.325418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 777/1545] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 18:06:18.326395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.341154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a0998-9dca-4997-b402-3150d68b4bc9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.353790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.354481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.354765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.355042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.355811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:18.356121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.357809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.377958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 769/1546] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:18.380229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.380793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.381294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.382161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 778/1547] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:18.389342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.410254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f223668-27dd-4d3a-a188-550315aa7578 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.424536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.426145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.541777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e63caa-e38b-4df6-a77d-1352b08c7bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.542414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e63caa-e38b-4df6-a77d-1352b08c7bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.542705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e63caa-e38b-4df6-a77d-1352b08c7bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.542989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e63caa-e38b-4df6-a77d-1352b08c7bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.543857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e63caa-e38b-4df6-a77d-1352b08c7bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.546078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e63caa-e38b-4df6-a77d-1352b08c7bd3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.560571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 770/1548] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.561558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.573772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e198e9b-06da-4149-9f85-a4f364e5a030 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.579584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c462a5b-ff3a-4930-81d1-a9c1773eabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.580418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c462a5b-ff3a-4930-81d1-a9c1773eabcc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.580712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c462a5b-ff3a-4930-81d1-a9c1773eabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.580994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c462a5b-ff3a-4930-81d1-a9c1773eabcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.581744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c462a5b-ff3a-4930-81d1-a9c1773eabcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.584170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c462a5b-ff3a-4930-81d1-a9c1773eabcc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.635495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 779/1549] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:06:18.638899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fb58-1911-4d91-aa46-15c29d29f705 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.638899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fb58-1911-4d91-aa46-15c29d29f705 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.638899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fb58-1911-4d91-aa46-15c29d29f705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.658975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dde8fb58-1911-4d91-aa46-15c29d29f705 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.660609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dde8fb58-1911-4d91-aa46-15c29d29f705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.670584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 780/1550] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.672743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.673035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.673444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.694816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.696204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.700159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 771/1551] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:18.701606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8159d0-0b1b-42e7-aa87-d344982bd32f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.701904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8159d0-0b1b-42e7-aa87-d344982bd32f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.702173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8159d0-0b1b-42e7-aa87-d344982bd32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.709443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 781/1552] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.710338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.721463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66791-9c20-4d05-8ded-b93ce11bd68d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.732226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5a8159d0-0b1b-42e7-aa87-d344982bd32f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.733497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5a8159d0-0b1b-42e7-aa87-d344982bd32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.789147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.789838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.790135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.790416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.791153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:18.791502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.793202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.807515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 782/1553] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:18.807753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.809888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 772/1554] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.815404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b79bd-a639-4380-8e67-fa80a34eb171 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.816155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b79bd-a639-4380-8e67-fa80a34eb171 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.816864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b79bd-a639-4380-8e67-fa80a34eb171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.829887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e1585-d7bb-4b6d-9499-8f893a7f99da admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.837089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b2b35-ab1c-407d-8857-c9155d2c8805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.838137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2b35-ab1c-407d-8857-c9155d2c8805 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.838620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2b35-ab1c-407d-8857-c9155d2c8805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.839102 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2b35-ab1c-407d-8857-c9155d2c8805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.840466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b2b35-ab1c-407d-8857-c9155d2c8805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.844062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b2b35-ab1c-407d-8857-c9155d2c8805 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.844740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c53b79bd-a639-4380-8e67-fa80a34eb171 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.846001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c53b79bd-a639-4380-8e67-fa80a34eb171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.854547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 773/1555] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.855920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.856220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.856483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.860785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 783/1556] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:18.865480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b282-d6e1-44c5-9a27-3505eb5fb8fb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.865775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b282-d6e1-44c5-9a27-3505eb5fb8fb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.866053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b282-d6e1-44c5-9a27-3505eb5fb8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.886949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.887643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e479b282-d6e1-44c5-9a27-3505eb5fb8fb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.888225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.889071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e479b282-d6e1-44c5-9a27-3505eb5fb8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.899007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 774/1557] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.899007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.908362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 784/1558] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.908362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.908362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.908362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.920789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca655f-c6fa-4210-a2b6-d2c123d3b0f2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.926997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.927791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.928080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.928359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.929098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.930131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:06:18.930131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:06:18.930131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:06:18.931356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe3409-91ee-4cf0-8a3c-c30bfaba4223 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.936744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:18.938852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:18.950652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 785/1559] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:18.951798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:18.964937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a1475-7085-4a4c-a99f-fc58553acfcb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:18.970624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:18.971302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:18.971624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:18.971907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:18.972609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:18.973644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:06:18.973644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:06:18.973644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:06:18.974873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:18.979215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:19.096612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 775/1560] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:19.098149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30868611-cfe6-4850-8f2d-045323406027 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.098454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30868611-cfe6-4850-8f2d-045323406027 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.099130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30868611-cfe6-4850-8f2d-045323406027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.113257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:19.119900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-30868611-cfe6-4850-8f2d-045323406027 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:19.121152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-30868611-cfe6-4850-8f2d-045323406027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:19.130249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 776/1561] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:19.131703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.131997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.132276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.152441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:19.153656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:19.169326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 777/1562] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:19.170569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.183287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f0880-087a-4b34-bc2b-1f259a0921ca None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.191899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa1107-dd8d-4a06-b959-1c64b16b34d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.195355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1107-dd8d-4a06-b959-1c64b16b34d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.195355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1107-dd8d-4a06-b959-1c64b16b34d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.195355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1107-dd8d-4a06-b959-1c64b16b34d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.195355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa1107-dd8d-4a06-b959-1c64b16b34d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.197856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa1107-dd8d-4a06-b959-1c64b16b34d2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.206281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 786/1563] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:18 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:19.209748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.213272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 778/1564] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:19.214650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.214971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.215228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.234878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:19.236397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:19.301181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4038f4-ddb4-4be3-8c3c-5c6da4a3baad admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.306183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.306875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.307166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.307487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.308232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:19.308546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.310561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.323339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 787/1565] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:19.324268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.339093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68189626-d816-4073-8a1f-d085d33ec68a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.347611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.348625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.349058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.349622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.349622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:19.349622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.352858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.383211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 788/1566] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:19.383211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.386282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7084dd4-ea7f-4cfb-bbe7-7213e5d24948 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.386476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 779/1567] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:19.387432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.392598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.393684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.394071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.394451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.395423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.397188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.399252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db78b77-dcd1-459a-ad8a-4210570f2101 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.404202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.404880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.405166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.405467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.410294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:19.410645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.415094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 789/1568] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:19.415300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.415446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.426608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebafc91-6829-425b-b805-422a329b3d52 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.427343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 780/1569] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:19.428690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.431845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.432551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.432822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.433101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.433836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:19.434147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.436172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.443769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1dc8-2e19-4635-a4f3-fc87a0a3ae26 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.449890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.450639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.450932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.451230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.452376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.452580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 790/1570] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:19.454571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.454743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.473267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ad235-df69-4769-a521-5efe40d7a75b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.478315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.479234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.479634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.479887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] PATH_INFO: `/v3/projects/7c39ef5406144d3787a865d9d003c686/users/bc0722ec1002496cb0ba7755b08bb492/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.496087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c39ef5406144d3787a865d9d003c686, user_id=bc0722ec1002496cb0ba7755b08bb492, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.497128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:06:19.497128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:06:19.497128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:06:19.499092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.543405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 781/1571] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:19.544257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.554768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce028ad-869a-49b6-983c-167727ee7178 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.559491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.560157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.560444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.560720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.561417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.564333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.571043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 791/1572] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:19 2022] PUT /identity/v3/projects/7c39ef5406144d3787a865d9d003c686/users/bc0722ec1002496cb0ba7755b08bb492/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:19.572449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.651332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb7c229-accb-4fcc-b006-ee931b92d5ed admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.651805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 782/1573] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:19.652670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.679486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:19.724407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.725820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.726056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.726679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.727893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.729590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905fdef-5d6b-4eaf-9e67-588347f39972 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.730945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.737361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.738035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.738318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.738595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.739321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:19.739660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.741275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.754140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 783/1574] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:19.755005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.766862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1eba7f-022d-4a60-b279-5749c936b052 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.771884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.772700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.772836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.773100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.773826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.775506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.787243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 784/1575] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:19.788087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.799553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59a429-9ca7-46d2-9dd3-9d2a6bb5d22d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.804374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.810062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.810062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.810062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.810062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.810062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.819388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:19.845860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 792/1576] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/projects => generated 385 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:19.845860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:19.913020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9feca66-a48c-4901-b2ca-e76bb308b86f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:19.917757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed153de-7d1c-45fe-a7c3-db29957368b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:19.918427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed153de-7d1c-45fe-a7c3-db29957368b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:19.918709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed153de-7d1c-45fe-a7c3-db29957368b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:19.918984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed153de-7d1c-45fe-a7c3-db29957368b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:19.919770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed153de-7d1c-45fe-a7c3-db29957368b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:19.922800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed153de-7d1c-45fe-a7c3-db29957368b5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:19.944992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 785/1577] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/projects => generated 389 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:19.945986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.016443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 793/1578] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:19 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.017037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fbfd3-dd2a-4f3d-9929-9d4f96af6ca0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.018009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.018305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.018820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.050316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] MFA Rules not processed for user `bc0722ec1002496cb0ba7755b08bb492`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:20.051915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:20.096722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.097784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.098210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.098559 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.099543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.101572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.114361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 786/1579] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:20.115424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.129706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35a810d-340c-4235-ad98-17a3bc10a026 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.135136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.135847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.136147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.136451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.137141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.139557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.146544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 794/1580] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:20.147665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.166915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4560470-0ef6-4eb9-88db-df30ffe26bf7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.166915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.167530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.167821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.168103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.168811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.170876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.258128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 787/1581] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.259111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.269778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c3fc-1492-434d-9382-df0dec59096a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.274466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.275387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.275701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.275976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] PATH_INFO: `/v3/projects/b566750533af4d0d96a3d0e0ffbc45ed/users/70d6f15465b84182a28d96d1808a9660/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.293211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b566750533af4d0d96a3d0e0ffbc45ed, user_id=70d6f15465b84182a28d96d1808a9660, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.295943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.304533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 795/1582] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.305367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.312384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:20.380739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18e265-f22b-4a37-b320-bce4ca4eec10 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.385529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.386489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.386489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.393278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.393278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.393278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.433125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 788/1583] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:20 2022] PUT /identity/v3/projects/b566750533af4d0d96a3d0e0ffbc45ed/users/70d6f15465b84182a28d96d1808a9660/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:20.433991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.496133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:20.508656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306e1f8-fe83-4c4a-944f-5da0b6b30a16 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.511346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 796/1584] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.512876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.518697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.519637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.520021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.520370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.521136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.523205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.530169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c853ca-b650-48c4-9aa0-551e145de62f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.538113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.539225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.539736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.540215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.541592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.545068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.619146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 789/1585] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.620044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.632699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ed36-d328-4423-9de2-296c81685487 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.638280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c6410f-d575-4b9d-83b2-f3b4f1ade9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.638970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6410f-d575-4b9d-83b2-f3b4f1ade9de admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.639260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6410f-d575-4b9d-83b2-f3b4f1ade9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.639607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6410f-d575-4b9d-83b2-f3b4f1ade9de admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.641015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6410f-d575-4b9d-83b2-f3b4f1ade9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.642757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c6410f-d575-4b9d-83b2-f3b4f1ade9de admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.658130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 790/1586] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:20.662714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 797/1587] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.662714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.663243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767c98-5736-4013-a728-cbf991f69919 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.663243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767c98-5736-4013-a728-cbf991f69919 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.663243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767c98-5736-4013-a728-cbf991f69919 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.663243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 791/1588] 10.222.0.58 () {54 vars in 961 bytes} [Thu Jun 16 18:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:06:20.664063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767c98-5736-4013-a728-cbf991f69919 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.678217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65392e7d-1734-4c45-b5f0-20cd4b5f74cc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.680833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767c98-5736-4013-a728-cbf991f69919 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.684126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b073-5516-4c3c-b4e0-fc2c5aa467d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.685257 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b073-5516-4c3c-b4e0-fc2c5aa467d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.685496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.685770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b073-5516-4c3c-b4e0-fc2c5aa467d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.686163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.686334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b073-5516-4c3c-b4e0-fc2c5aa467d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.686475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.686705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.701073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704b073-5516-4c3c-b4e0-fc2c5aa467d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.701293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.701377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704b073-5516-4c3c-b4e0-fc2c5aa467d5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.701497 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.705439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 798/1589] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:20.706820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.707115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.707452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.727569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:20.728804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:20.737305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 799/1590] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:20.742119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.748047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb87c95-dfb2-42d6-bbc3-bdcee1248405 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.752688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.753402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.753651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.753923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.754672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.756273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.773570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 800/1591] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:20.774460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.785575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873be015-44f9-4d35-ab43-f799f12cb22f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.792299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.793030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.793436 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.793720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.794459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.795892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 792/1592] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:20.796111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.799713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.813553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 801/1593] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:20.813819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5700e-2f5a-49e0-80dc-e130225b48b0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.814002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.814811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.815752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.816035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.816670 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] PATH_INFO: `/v3/projects/285d8bae36f34e6a80a3982ebb097ea3/users/9eac1fb288764b74baf30892237b3cdc/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.821539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcace31-9e1b-49ab-a829-a383d18f8644 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:20.826216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d8a44-11c0-474c-96da-e4fe3638307a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:20.826906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d8a44-11c0-474c-96da-e4fe3638307a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.828287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d8a44-11c0-474c-96da-e4fe3638307a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.828287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d8a44-11c0-474c-96da-e4fe3638307a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.829173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d8a44-11c0-474c-96da-e4fe3638307a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.831321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d8a44-11c0-474c-96da-e4fe3638307a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.836420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285d8bae36f34e6a80a3982ebb097ea3, user_id=9eac1fb288764b74baf30892237b3cdc, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:20.841760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:20.842934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 802/1594] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:20.845076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:20.845381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:20.845652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:20.864763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:20.865994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:20.930638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 793/1595] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:20 2022] PUT /identity/v3/projects/285d8bae36f34e6a80a3982ebb097ea3/users/9eac1fb288764b74baf30892237b3cdc/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:20.930946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.952361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 803/1596] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Jun 16 18:06:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:20.953932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:20.976598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:21.023647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c08db8-f2f4-45be-9213-6b204c3a82b8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.029288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.031476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407ff47-c141-4652-b5cd-3cde557cebac None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.033298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.033298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.033298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.033298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.033298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.045113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 794/1597] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:21.046079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.059065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ae7e9-57fb-4300-8388-84b0c603ec8f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.066563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.067304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.067641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.067922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.068661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.070266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.082530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 795/1598] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:21.083600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.093940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba810-28c8-45c4-bc3f-746a86016bda admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.099471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.100437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.100437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.100713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.101415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.102253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.103028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.103206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.103539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.103945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.115274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 796/1599] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:21.115539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.116776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.119107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.128931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 804/1600] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:21.129173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.134181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c562-ef50-4928-842e-bca06b315a44 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.141226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345bc35-c940-4cb2-91d5-be204167452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.141927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345bc35-c940-4cb2-91d5-be204167452e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.142114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9ec0-413e-4ccc-91ba-42cc3e5f890c service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.142235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345bc35-c940-4cb2-91d5-be204167452e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.142476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345bc35-c940-4cb2-91d5-be204167452e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.143273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345bc35-c940-4cb2-91d5-be204167452e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.144929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345bc35-c940-4cb2-91d5-be204167452e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.149874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.150834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.151117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.151430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.152184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.153849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.160810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 797/1601] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:21.162287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.162587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.162867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.167456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 805/1602] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:21.168333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.186704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:21.188290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:21.196364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 798/1603] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:21.197270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.220482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9762d-96ed-479c-98fe-d0173f819eed None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.225281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.226213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.226499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.226784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] PATH_INFO: `/v3/projects/ac7543e92c9b46c2b836bc5c33c43f40/users/5e7f13b5f714413396ead1663f9ffc50/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.252058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7543e92c9b46c2b836bc5c33c43f40, user_id=5e7f13b5f714413396ead1663f9ffc50, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.255056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.278165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499040b-3139-40da-abf5-f6ef29f8c9a4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.288281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.289575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.290130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.290602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.354252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 799/1604] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:21 2022] PUT /identity/v3/projects/ac7543e92c9b46c2b836bc5c33c43f40/users/5e7f13b5f714413396ead1663f9ffc50/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:21.355464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.373575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.376298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.385256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:21.386101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:21.391970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 806/1605] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:21.392871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.404311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcc254-5132-4f24-ba8c-0d73efe94d19 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.440441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23034243-4ebd-45ed-8454-3ca184ace36e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.445707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.453401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.453401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.453401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] PATH_INFO: `/v3/projects/5ad7a8c562934b3c843b10d79f31fee5/users/8498989e673d445dbc70aa9df55f391b/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.470165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad7a8c562934b3c843b10d79f31fee5, user_id=8498989e673d445dbc70aa9df55f391b, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.473364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.516394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.517118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.517407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.517726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.518557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.520414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.537259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 807/1606] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:21.537962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.554166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a7a24-a747-485e-87db-f5cbfb959461 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.561058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.563150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.563513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.563798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] PATH_INFO: `/v3/projects/11089493792047939c9f45f632a0ce05/users/30ac12728e8f4ec9af534612030d75eb/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.587779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 800/1607] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:21 2022] PUT /identity/v3/projects/5ad7a8c562934b3c843b10d79f31fee5/users/8498989e673d445dbc70aa9df55f391b/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:21.587779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.588120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11089493792047939c9f45f632a0ce05, user_id=30ac12728e8f4ec9af534612030d75eb, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.594036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.668073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fca92-dc5c-42b7-8860-0d08a938e816 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.675736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f743bc-e1f6-4c70-8b1a-593636a42bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.675736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f743bc-e1f6-4c70-8b1a-593636a42bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.675736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f743bc-e1f6-4c70-8b1a-593636a42bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.676217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f743bc-e1f6-4c70-8b1a-593636a42bdc admin admin] PATH_INFO: `/v3/projects/14b0926322074e2da25ddaae611d89d4/users/712089cbbf9a424fa5d4412ced2cdc58/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.678524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 808/1608] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:21 2022] PUT /identity/v3/projects/11089493792047939c9f45f632a0ce05/users/30ac12728e8f4ec9af534612030d75eb/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:21.683469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.696508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f743bc-e1f6-4c70-8b1a-593636a42bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b0926322074e2da25ddaae611d89d4, user_id=712089cbbf9a424fa5d4412ced2cdc58, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.700061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f743bc-e1f6-4c70-8b1a-593636a42bdc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.751418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd6262-4847-4a08-adca-ee515a7eb2a9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.755702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:21.756747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.757230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.757727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.803663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 801/1609] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:21 2022] PUT /identity/v3/projects/14b0926322074e2da25ddaae611d89d4/users/712089cbbf9a424fa5d4412ced2cdc58/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:21.806320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8492338-267c-4edb-a8df-b38ad20e6279 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:21.806933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8492338-267c-4edb-a8df-b38ad20e6279 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:21.808517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8492338-267c-4edb-a8df-b38ad20e6279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:21.901557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d8492338-267c-4edb-a8df-b38ad20e6279 None None] MFA Rules not processed for user `5e7f13b5f714413396ead1663f9ffc50`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:21.903746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d8492338-267c-4edb-a8df-b38ad20e6279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:21.910586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:21.916470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:21.931680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 809/1610] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:21.931680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:21.956223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a86977-1d4f-4583-b132-a2ba3d12072e service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:21.986698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:22.053717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 802/1611] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:22.055524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1c3f2-aef3-4bb7-ad82-4b91832e9e95 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.055822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1c3f2-aef3-4bb7-ad82-4b91832e9e95 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.056166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1c3f2-aef3-4bb7-ad82-4b91832e9e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.067671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.069394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.069912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.070500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] PATH_INFO: `/v3/projects/d590d2f6ec3249e2b847d696886203e5/users/db6419c987a745239d1738908391e2de/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.088237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-19f1c3f2-aef3-4bb7-ad82-4b91832e9e95 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:22.089125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-19f1c3f2-aef3-4bb7-ad82-4b91832e9e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:22.100240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d590d2f6ec3249e2b847d696886203e5, user_id=db6419c987a745239d1738908391e2de, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.104227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.107831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 803/1612] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:22.111828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ee50a-589f-42d2-bfd9-d10b3ecaf182 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.112125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ee50a-589f-42d2-bfd9-d10b3ecaf182 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.112436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ee50a-589f-42d2-bfd9-d10b3ecaf182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.132999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1e0ee50a-589f-42d2-bfd9-d10b3ecaf182 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:22.134245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1e0ee50a-589f-42d2-bfd9-d10b3ecaf182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:22.143622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 804/1613] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:22.145257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582560df-fe66-40db-9e00-972a3ffac384 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.145765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582560df-fe66-40db-9e00-972a3ffac384 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.146248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582560df-fe66-40db-9e00-972a3ffac384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.154330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:22.169556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-582560df-fe66-40db-9e00-972a3ffac384 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:22.171083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-582560df-fe66-40db-9e00-972a3ffac384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:22.190820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 805/1614] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:22.190820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a32dca-8822-47d2-98e1-f536376fcca9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.190820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a32dca-8822-47d2-98e1-f536376fcca9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.190820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a32dca-8822-47d2-98e1-f536376fcca9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.192704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 806/1615] 10.222.0.58 () {54 vars in 961 bytes} [Thu Jun 16 18:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:06:22.194065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a32dca-8822-47d2-98e1-f536376fcca9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.205985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 810/1616] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:21 2022] PUT /identity/v3/projects/d590d2f6ec3249e2b847d696886203e5/users/db6419c987a745239d1738908391e2de/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:22.207793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.286343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a32dca-8822-47d2-98e1-f536376fcca9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:22.290870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.291553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.291839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.292110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.301022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4722-0755-45f7-a694-c85a67e02f13 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:22.305244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c30cc1-0011-4650-870b-8fe15fd813b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.306038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c30cc1-0011-4650-870b-8fe15fd813b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.306396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c30cc1-0011-4650-870b-8fe15fd813b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.306682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c30cc1-0011-4650-870b-8fe15fd813b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.395409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c30cc1-0011-4650-870b-8fe15fd813b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.397324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c30cc1-0011-4650-870b-8fe15fd813b7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.401649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.403646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.414026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 807/1617] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:22.416205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 811/1618] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:22.416517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.426529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11967011-9274-43b6-8b2b-884fbe4370e8 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:22.430557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.430723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e4a7d-d753-444b-8f3b-2829fa820d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.431009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.431463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e4a7d-d753-444b-8f3b-2829fa820d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.431611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.431759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e4a7d-d753-444b-8f3b-2829fa820d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.432336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e4a7d-d753-444b-8f3b-2829fa820d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.444114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e4a7d-d753-444b-8f3b-2829fa820d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.447499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e4a7d-d753-444b-8f3b-2829fa820d29 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.454762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:22.456161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:22.457946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 808/1619] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:22.459288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.459645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.460066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.517140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] MFA Rules not processed for user `db6419c987a745239d1738908391e2de`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:22.519408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:22.547891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:22.597509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 812/1620] 10.222.0.58 () {58 vars in 1058 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:22.598534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.618865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6b6fd-7ef9-49c8-9af6-205aa1e9bb5f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:22.626103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48585498-d479-40ab-9693-3bb915edc77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.626780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48585498-d479-40ab-9693-3bb915edc77e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.627068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48585498-d479-40ab-9693-3bb915edc77e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.627349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48585498-d479-40ab-9693-3bb915edc77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.640475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48585498-d479-40ab-9693-3bb915edc77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.644089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48585498-d479-40ab-9693-3bb915edc77e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.667049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 813/1621] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:22.672846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b079886-50cf-4e72-9e02-ef21f29328df None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.673147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b079886-50cf-4e72-9e02-ef21f29328df None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.673417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b079886-50cf-4e72-9e02-ef21f29328df None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.674862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 814/1622] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jun 16 18:06:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:06:22.682540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.683219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.683804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.696066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 809/1623] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:22.697432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.710461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef23c-e460-412c-bcdc-ee330bedbb50 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:22.717461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9aee-417c-4f74-8058-f199d4517dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.718834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9aee-417c-4f74-8058-f199d4517dda admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.719258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9aee-417c-4f74-8058-f199d4517dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.719648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9aee-417c-4f74-8058-f199d4517dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.720390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c9aee-417c-4f74-8058-f199d4517dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.722397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c9aee-417c-4f74-8058-f199d4517dda admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.741624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:22.742918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:22.765703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 815/1624] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:22.788002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.800591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25197893-84cd-44e5-914b-ddbe42e6549b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:22.804954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:22.805630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:22.806455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:22.806744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:22.828746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:22.831179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:22.861956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 816/1625] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:22.881896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 810/1626] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:22.882704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:22.898036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d8dfe-b4a5-4b31-8126-ae3328d375a5 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.023441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.024547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.025118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.025667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.026600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.029967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.056607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52103c3-99a9-4e48-83a0-e41677fbb4e0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.056837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52103c3-99a9-4e48-83a0-e41677fbb4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.057094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52103c3-99a9-4e48-83a0-e41677fbb4e0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.058747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 811/1627] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jun 16 18:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:06:23.079637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.080147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.080147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.102974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:23.117976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:23.120097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:23.152923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 817/1628] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:22 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:06:23.157399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.182557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcde5c-3a23-4691-8202-610b19059dc2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.201839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.205838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.205838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.205838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.209467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.220082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.237572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 818/1629] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:06:23.243629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.268316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70832f5e-f85e-40eb-aee8-f13b0add38d7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.277778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.279146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.279665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.280326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] PATH_INFO: `/v3/projects/f7e9ff3fdea14fa8afd51fddc3a8d0da/users/23d1a6c1cbe04b36826ae26e9133c0a7/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.313553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 812/1630] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:06:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:23.348877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.362744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e9ff3fdea14fa8afd51fddc3a8d0da, user_id=23d1a6c1cbe04b36826ae26e9133c0a7, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.368882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.466075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b7f53-9cf2-4993-bf9c-55ca2ec65768 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.475858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.476566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.476986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.477432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.489842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 819/1631] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:23 2022] PUT /identity/v3/projects/f7e9ff3fdea14fa8afd51fddc3a8d0da/users/23d1a6c1cbe04b36826ae26e9133c0a7/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:23.493995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.497598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.501600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.524360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 813/1632] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:06:23.617675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541be3f2-9baa-48d2-a839-b61dc987b65f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.631418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.631418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.631418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.631418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.631991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.631991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.648415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.648635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 820/1633] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:06:23.667532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cbce1-363b-488b-bcd5-cd5cca3572e3 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.677919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178dc59-1e92-4535-8f7e-8a5adee090ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.678985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178dc59-1e92-4535-8f7e-8a5adee090ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.679343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178dc59-1e92-4535-8f7e-8a5adee090ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.679723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178dc59-1e92-4535-8f7e-8a5adee090ed admin admin] PATH_INFO: `/v3/projects/f7e9ff3fdea14fa8afd51fddc3a8d0da/users/23d1a6c1cbe04b36826ae26e9133c0a7/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.691989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178dc59-1e92-4535-8f7e-8a5adee090ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e9ff3fdea14fa8afd51fddc3a8d0da, user_id=23d1a6c1cbe04b36826ae26e9133c0a7, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.696639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.705969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178dc59-1e92-4535-8f7e-8a5adee090ed admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.710345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:23.716776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541955c-bac7-465c-850b-442d5d8f07a4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:23.731412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 814/1634] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:23 2022] PUT /identity/v3/projects/f7e9ff3fdea14fa8afd51fddc3a8d0da/users/23d1a6c1cbe04b36826ae26e9133c0a7/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:23.745949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.746514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.748967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.794699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] MFA Rules not processed for user `23d1a6c1cbe04b36826ae26e9133c0a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:23.794699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:23.896467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:23.898281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:23.898712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:23.899346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] PATH_INFO: `/v3/users/bc0722ec1002496cb0ba7755b08bb492` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:23.901861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc0722ec1002496cb0ba7755b08bb492)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:23.903130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:06:23.903130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jun 16 18:06:23.903130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: Jun 16 18:06:23.907551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:23.936544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 815/1635] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:23 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:23.950147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:23.973861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:24.024457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640b4f1-ebe7-4728-bd96-4e5222f61cba None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:24.028850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:24.029597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:24.029931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:24.030357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:24.040695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:24.043927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:24.055172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 816/1636] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:23 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:06:24.210308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:24.220617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb9ed3-0e9c-4399-a230-85bbdee5ca37 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:24.221158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc0722ec1002496cb0ba7755b08bb492'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:24.222695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc0722ec1002496cb0ba7755b08bb492'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:24.226186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505c58a-febc-43f9-880a-4168c7bc295c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:24.227079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505c58a-febc-43f9-880a-4168c7bc295c service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:24.227611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505c58a-febc-43f9-880a-4168c7bc295c service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:24.228068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505c58a-febc-43f9-880a-4168c7bc295c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:24.323554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc0722ec1002496cb0ba7755b08bb492'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:24.326554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505c58a-febc-43f9-880a-4168c7bc295c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:24.329943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505c58a-febc-43f9-880a-4168c7bc295c service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:24.345790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc0722ec1002496cb0ba7755b08bb492'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:24.355509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 817/1637] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:06:24.410229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc0722ec1002496cb0ba7755b08bb492'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:24.421956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 821/1638] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:23 2022] DELETE /identity/v3/users/bc0722ec1002496cb0ba7755b08bb492 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:24.424721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:24.425283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:24.425776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:24.575769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:24.575769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:24.612180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 818/1639] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:24.627238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:24.629180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:24.735779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21d38a-2124-4153-bb75-a3ffda673035 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:24.740026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d04397-1824-4943-9527-e7549089cb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:24.740719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d04397-1824-4943-9527-e7549089cb36 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:24.741011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d04397-1824-4943-9527-e7549089cb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:24.741324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d04397-1824-4943-9527-e7549089cb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:24.753877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d04397-1824-4943-9527-e7549089cb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:25.043156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d04397-1824-4943-9527-e7549089cb36 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:25.043156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 822/1640] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:25.916589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:25.919416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:25.954867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9a403-f7ee-4b52-92e6-4bd5d28f5197 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:25.970353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:25.972545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:25.973210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:25.973825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] PATH_INFO: `/v3/projects/7c39ef5406144d3787a865d9d003c686` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:25.976619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c39ef5406144d3787a865d9d003c686)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:25.979870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:06:25.979870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 16 18:06:25.979870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: Jun 16 18:06:25.983160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:26.203988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c39ef5406144d3787a865d9d003c686', 'request_id': 'req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:26.205936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:26.275143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c39ef5406144d3787a865d9d003c686', 'request_id': 'req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:26.307996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c39ef5406144d3787a865d9d003c686', 'request_id': 'req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:26.339866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] The token cache is being invalidate because project 7c39ef5406144d3787a865d9d003c686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:26.342154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:26.343844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d890f24-fa8e-4827-b4a6-8a88ee2169b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:26.348820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 819/1641] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:25 2022] DELETE /identity/v3/projects/7c39ef5406144d3787a865d9d003c686 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:26.354571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:26.355552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:26.356128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:26.393676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:26.395330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:26.430555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:26.541023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 823/1642] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:26.543032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053b982-33b9-4480-b88b-aa092772825f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:26.543706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053b982-33b9-4480-b88b-aa092772825f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:26.545100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053b982-33b9-4480-b88b-aa092772825f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:26.571259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d053b982-33b9-4480-b88b-aa092772825f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:26.572915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d053b982-33b9-4480-b88b-aa092772825f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:26.586687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 820/1643] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:26.588886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:26.611059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7845d-1575-4949-8c9b-a44389add095 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:26.621440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:26.622158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:26.624130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:26.624832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:26.626323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:26.626796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:26.632499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:26.660547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 824/1644] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:26.665084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053b982-33b9-4480-b88b-aa092772825f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:26.677834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053b982-33b9-4480-b88b-aa092772825f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:26.686648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:26.688519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:26.689624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:26.689941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:26.690970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:26.693671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:26.831020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 821/1645] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:26 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:26.958209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:26.958209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d6082-863c-4cad-a2d7-960594dbc360 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:27.010974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:27.012033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:27.012326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:27.012604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:27.013365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:27.017058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:27.128364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 825/1646] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:26 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:27.132682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:27.151155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859c2b6-f4d9-4c44-9022-219450a81cb9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:27.171060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:27.171060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:27.171060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:27.171060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:27.171480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:27.171480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:27.197317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:27.207496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 822/1647] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:27.210986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f4df6-1f2f-4824-97e3-5a74313d37ac admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:27.222946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa439617-9dd9-4699-a416-8df6498d4814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:27.235745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa439617-9dd9-4699-a416-8df6498d4814 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:27.240783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa439617-9dd9-4699-a416-8df6498d4814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:27.241377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa439617-9dd9-4699-a416-8df6498d4814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:27.242422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa439617-9dd9-4699-a416-8df6498d4814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:27.244321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa439617-9dd9-4699-a416-8df6498d4814 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:27.311922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:27.317076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 826/1648] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:27.330450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d157e-439b-4397-8420-0e0caf74aa99 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:27.344244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:27.347749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:27.348179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:27.348914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] PATH_INFO: `/v3/projects/7842ac120a5a401e9388f8daf5880cf5/users/fe304332ce8a469ea259ee718192004a/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:27.372640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7842ac120a5a401e9388f8daf5880cf5, user_id=fe304332ce8a469ea259ee718192004a, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:27.376493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:27.481835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 823/1649] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:27 2022] PUT /identity/v3/projects/7842ac120a5a401e9388f8daf5880cf5/users/fe304332ce8a469ea259ee718192004a/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:27.496417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:27.497187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:27.497187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:27.559233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] MFA Rules not processed for user `fe304332ce8a469ea259ee718192004a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:27.560776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:27.710699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 827/1650] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:27.712233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:27.806108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e089bf-f496-4902-885c-fca21c0e4c58 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:27.813592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:27.814485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:27.815130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:27.815595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:27.816799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:27.820135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:27.951709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 824/1651] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:27 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:27.957542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:27.973316 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ee1e-44a9-4cbf-a51f-2c7798f28975 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:28.125093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:28.125914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:28.126212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:28.126488 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:28.127440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:28.129615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:28.221432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 828/1652] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:27 2022] POST /identity/v3/users => generated 300 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:28.225826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:28.242805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0ce8e-f78e-48de-b169-f60fec9dba72 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:28.249625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:28.258298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:28.262780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:28.262780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:28.262780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:28.262780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:28.262780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:28.275405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 825/1653] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:28.279390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:28.293862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc98aa2-95c4-4514-964b-23ac47b25404 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:28.300782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:28.301833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:28.302195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:28.302544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] PATH_INFO: `/v3/projects/cb3cce3222cb4c56b4f9c908dac362b4/users/54454513f622495bad1902d4763f6e79/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:28.329689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3cce3222cb4c56b4f9c908dac362b4, user_id=54454513f622495bad1902d4763f6e79, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:28.332480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:28.421450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:28.423064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 829/1654] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:28 2022] PUT /identity/v3/projects/cb3cce3222cb4c56b4f9c908dac362b4/users/54454513f622495bad1902d4763f6e79/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:28.446264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c5fa-8237-4bd2-882b-e3f82836a6ef admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:28.675501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:28.696963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8f882-326c-441d-8b6b-32c4983096b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:28.697663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f882-326c-441d-8b6b-32c4983096b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:28.700323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f882-326c-441d-8b6b-32c4983096b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:28.706007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8f882-326c-441d-8b6b-32c4983096b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:28.706007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8f882-326c-441d-8b6b-32c4983096b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:28.706007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8f882-326c-441d-8b6b-32c4983096b1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:28.740386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 826/1655] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:06:28.742831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:28.783105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad739e2-697d-4b90-9f77-228c05f50d4c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:28.807826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:28.809409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:28.809924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:28.810788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] PATH_INFO: `/v3/projects/cb3cce3222cb4c56b4f9c908dac362b4/users/54454513f622495bad1902d4763f6e79/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:28.819770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3cce3222cb4c56b4f9c908dac362b4, user_id=54454513f622495bad1902d4763f6e79, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:28.824595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:28.868662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 830/1656] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:28 2022] PUT /identity/v3/projects/cb3cce3222cb4c56b4f9c908dac362b4/users/54454513f622495bad1902d4763f6e79/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:06:28.869898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:28.870258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:28.870549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:28.962336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] MFA Rules not processed for user `54454513f622495bad1902d4763f6e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:28.966828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:29.118190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 827/1657] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:29.127596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:29.135300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:29.374489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af0a2-b214-4185-809d-5533f5f41d20 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:29.383467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5925005-a7af-42d2-b68b-d9356783764f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:29.384151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925005-a7af-42d2-b68b-d9356783764f service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:29.387423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925005-a7af-42d2-b68b-d9356783764f service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:29.387423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925005-a7af-42d2-b68b-d9356783764f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:29.422957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5925005-a7af-42d2-b68b-d9356783764f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:29.424466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5925005-a7af-42d2-b68b-d9356783764f service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:29.459670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 831/1658] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:29.517068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:29.738633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45455e-c3cb-45a0-b9f5-4f19f50a983e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:29.750507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:29.751604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:29.751829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:29.753233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] PATH_INFO: `/v3/users/fe304332ce8a469ea259ee718192004a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:29.754952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe304332ce8a469ea259ee718192004a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:29.759439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:30.099903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe304332ce8a469ea259ee718192004a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:30.130551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe304332ce8a469ea259ee718192004a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:30.171937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:30.179807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe304332ce8a469ea259ee718192004a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:30.274532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe304332ce8a469ea259ee718192004a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:30.275185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe304332ce8a469ea259ee718192004a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:30.284856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 828/1659] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:29 2022] DELETE /identity/v3/users/fe304332ce8a469ea259ee718192004a => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:30.289472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:30.289966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:30.289966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:30.344461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:30.345779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:30.537197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 832/1660] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:30.571702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:30.588278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:30.756660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c438fff-fd43-4d2f-bdc2-2febb4569ace admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:30.772413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0e98b-fe48-4e0b-bbb7-582a33fdad08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:30.773167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e98b-fe48-4e0b-bbb7-582a33fdad08 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:30.773462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e98b-fe48-4e0b-bbb7-582a33fdad08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:30.773740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0e98b-fe48-4e0b-bbb7-582a33fdad08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:30.808531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0e98b-fe48-4e0b-bbb7-582a33fdad08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:30.810018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0e98b-fe48-4e0b-bbb7-582a33fdad08 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:30.831781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 829/1661] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:32.049684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:32.051310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:32.157593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b2b67-f55a-45e0-8157-05f37559d57f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:32.157593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:32.157593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:32.157593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:32.157862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] PATH_INFO: `/v3/users/db6419c987a745239d1738908391e2de` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:32.157862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6419c987a745239d1738908391e2de)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:32.157862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:32.157862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:32.361492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6419c987a745239d1738908391e2de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:32.363302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6419c987a745239d1738908391e2de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:32.450467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6419c987a745239d1738908391e2de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:32.469139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6419c987a745239d1738908391e2de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:32.518152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6419c987a745239d1738908391e2de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:32.530849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 833/1662] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:32 2022] DELETE /identity/v3/users/db6419c987a745239d1738908391e2de => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:32.537476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:32.537476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:32.538140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:32.597880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:32.597880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:32.712769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 830/1663] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:32.739795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:32.741430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:32.809083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:32.810604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:32.869445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee84c4c-3656-4fce-9209-459fe2c70615 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:32.884465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:32.885377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:32.885788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:32.886080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] PATH_INFO: `/v3/projects/7842ac120a5a401e9388f8daf5880cf5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:32.887909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9b19f-544f-452a-8dbb-1ef3bc6077b5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:32.890026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=7842ac120a5a401e9388f8daf5880cf5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:32.892355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:32.900610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:32.901295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:32.901580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:32.901853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:32.936953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:32.938450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:32.982703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 834/1664] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:33.073788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7842ac120a5a401e9388f8daf5880cf5', 'request_id': 'req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.196884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7842ac120a5a401e9388f8daf5880cf5', 'request_id': 'req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.205674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7842ac120a5a401e9388f8daf5880cf5', 'request_id': 'req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.234485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] The token cache is being invalidate because project 7842ac120a5a401e9388f8daf5880cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:33.237265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.249802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.256536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 831/1665] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:32 2022] DELETE /identity/v3/projects/7842ac120a5a401e9388f8daf5880cf5 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:06:33.261327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:33.294742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb44c6f-d02b-40c6-a79f-a3e9537541c8 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:33.470866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:33.477620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:33.477620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:33.477620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] PATH_INFO: `/v3/users/54454513f622495bad1902d4763f6e79` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:33.477620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=54454513f622495bad1902d4763f6e79)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:33.480195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:33.727580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:33.744474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54454513f622495bad1902d4763f6e79'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.745532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54454513f622495bad1902d4763f6e79'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.815240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54454513f622495bad1902d4763f6e79'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.836341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54454513f622495bad1902d4763f6e79'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.890589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b3b5ef5-debe-4da1-b0f4-50bc42c75bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54454513f622495bad1902d4763f6e79'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:33.898977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 835/1666] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:33 2022] DELETE /identity/v3/users/54454513f622495bad1902d4763f6e79 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:34.555532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:34.557282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:34.585722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efea8c-7084-4af7-bb3c-b56f9d32b5ed admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:34.603894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:34.660367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:34.661492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:34.661899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:34.662287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] PATH_INFO: `/v3/projects/d590d2f6ec3249e2b847d696886203e5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:34.665138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d590d2f6ec3249e2b847d696886203e5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:34.669702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:34.721963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a184b73-c405-48b4-b256-df83a80d7f6f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:34.722796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a184b73-c405-48b4-b256-df83a80d7f6f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:34.723242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a184b73-c405-48b4-b256-df83a80d7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:34.774851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8a184b73-c405-48b4-b256-df83a80d7f6f None None] MFA Rules not processed for user `70d6f15465b84182a28d96d1808a9660`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:34.776123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8a184b73-c405-48b4-b256-df83a80d7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:34.853948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd590d2f6ec3249e2b847d696886203e5', 'request_id': 'req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:34.959952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd590d2f6ec3249e2b847d696886203e5', 'request_id': 'req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:34.972332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd590d2f6ec3249e2b847d696886203e5', 'request_id': 'req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:34.982293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 836/1667] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:34.984911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:34.986355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:34.986641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:35.021941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] The token cache is being invalidate because project d590d2f6ec3249e2b847d696886203e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:35.022257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.024667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.028838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 832/1668] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:34 2022] DELETE /identity/v3/projects/d590d2f6ec3249e2b847d696886203e5 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:35.034124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:35.035709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:35.047175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] MFA Rules not processed for user `70d6f15465b84182a28d96d1808a9660`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:35.054727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:35.208267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 837/1669] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:35.209086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9279cc-7b1b-4797-815e-e3afd75dcbf4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:35.230745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:35.232817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:35.233627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:35.235149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:35.236616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:35.238642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] PATH_INFO: `/v3/users/23d1a6c1cbe04b36826ae26e9133c0a7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:35.241271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d1a6c1cbe04b36826ae26e9133c0a7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:35.256884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:35.429101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce63cb-e785-4eca-be73-e771b973599b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:35.445333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:35.446016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:35.446469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:35.446557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:35.451926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:35.481874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:35.484849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:35.509884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 838/1670] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:35.510804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:35.522827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:35.554336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975756-fa76-463e-bda3-9f594b7bc142 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:35.569443 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d1a6c1cbe04b36826ae26e9133c0a7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.570079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:35.570572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:35.570863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:35.571137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] PATH_INFO: `/v3/projects/cb3cce3222cb4c56b4f9c908dac362b4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:35.577641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3cce3222cb4c56b4f9c908dac362b4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:35.579862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:35.642328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d1a6c1cbe04b36826ae26e9133c0a7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.674348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d1a6c1cbe04b36826ae26e9133c0a7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.754584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3cce3222cb4c56b4f9c908dac362b4', 'request_id': 'req-fee1bb7a-226b-47f4-b975-91421033de7f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.772895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3cce3222cb4c56b4f9c908dac362b4', 'request_id': 'req-fee1bb7a-226b-47f4-b975-91421033de7f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.783014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d1a6c1cbe04b36826ae26e9133c0a7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.783966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2b313dfb-2e4e-454e-b4b4-82b1e5888319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d1a6c1cbe04b36826ae26e9133c0a7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.786738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3cce3222cb4c56b4f9c908dac362b4', 'request_id': 'req-fee1bb7a-226b-47f4-b975-91421033de7f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.795434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 833/1671] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:35 2022] DELETE /identity/v3/users/23d1a6c1cbe04b36826ae26e9133c0a7 => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:06:35.875113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] The token cache is being invalidate because project cb3cce3222cb4c56b4f9c908dac362b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:35.875659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.877766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fee1bb7a-226b-47f4-b975-91421033de7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:35.878613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 839/1672] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:35 2022] DELETE /identity/v3/projects/cb3cce3222cb4c56b4f9c908dac362b4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:35.886261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:35.886593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:35.886880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:35.926965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:35.931687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:35.960205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:36.086146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 834/1673] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:36.096274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.096567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.096957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.140336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:36.142330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:36.164205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 840/1674] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:36.169432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:36.196957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c643f7-c48d-4e91-aacb-19364cbd9033 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:36.215152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:36.215902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.216230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.216461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.218003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:36.218125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:36.230402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:36.271166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 835/1675] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:36.272342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:36.285970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e300e-f11c-4b39-b863-1037fa52e4ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:36.300591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:36.301935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.302374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.302835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.304193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:36.311270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:36.441193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 841/1676] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:36 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:36.442710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:36.463664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75674e0b-57bc-463a-874a-1b09330ad2ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:36.597393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:36.600422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.600898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.601761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.603280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:36.605800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:36.753780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 836/1677] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:36 2022] POST /identity/v3/users => generated 291 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:36.755207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:36.769473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507ec9f-1552-4c08-b8c5-d99a9a03ca8e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:36.782841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f512dd-3e62-4b50-8322-de21117fb9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:36.784429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f512dd-3e62-4b50-8322-de21117fb9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.784863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f512dd-3e62-4b50-8322-de21117fb9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.785283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f512dd-3e62-4b50-8322-de21117fb9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.786203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f512dd-3e62-4b50-8322-de21117fb9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:36.788153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f512dd-3e62-4b50-8322-de21117fb9e5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:36.822348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 842/1678] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:36.826758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:36.835945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c457b2-a91b-4103-b4ff-0ac6a87da257 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.836389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c457b2-a91b-4103-b4ff-0ac6a87da257 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.836679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c457b2-a91b-4103-b4ff-0ac6a87da257 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.838413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 843/1679] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jun 16 18:06:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:06:36.847410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.847710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.847996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.850671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29626545-4ce7-4258-b83a-28b8167c97f5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:36.871796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:36.872495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.872792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.873165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:36.873918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:36.875570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:36.903039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 837/1680] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:36.904034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:36.933324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0d069-316d-4754-a2ea-0f09a9d73bc2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:36.934317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] MFA Rules not processed for user `6aae965af8284363820254c9286adf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:36.936227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:36.953412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f8569-45b7-4717-9bea-523c8b40a8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:36.954586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f8569-45b7-4717-9bea-523c8b40a8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:36.955017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f8569-45b7-4717-9bea-523c8b40a8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:36.955496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f8569-45b7-4717-9bea-523c8b40a8c0 admin admin] PATH_INFO: `/v3/projects/e8a415c2319f4a57abc14cb56e40ecef/users/72c29f36e9614fb4905fdfd34b1859ba/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:37.001301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f8569-45b7-4717-9bea-523c8b40a8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a415c2319f4a57abc14cb56e40ecef, user_id=72c29f36e9614fb4905fdfd34b1859ba, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:37.004978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f8569-45b7-4717-9bea-523c8b40a8c0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:37.064828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 844/1681] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:06:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:37.119860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 838/1682] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:36 2022] PUT /identity/v3/projects/e8a415c2319f4a57abc14cb56e40ecef/users/72c29f36e9614fb4905fdfd34b1859ba/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:37.130042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:37.132110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:37.132407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:37.132689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:37.135151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:37.155538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142f6fd-e636-4109-8e73-6afbdfe3c967 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:37.200761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] MFA Rules not processed for user `72c29f36e9614fb4905fdfd34b1859ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:37.202793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:37.236781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:37.372696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:37.373761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:37.380743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:37.381437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:37.390537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 839/1683] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:37.392110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:37.401533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:37.407013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:37.429881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 845/1684] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:37.563523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e210b54-04b9-4780-a955-b8858bd64c9c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:37.583875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:37.583875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:37.583875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:37.583875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:37.584486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:37.584486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:37.720434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 840/1685] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:37 2022] POST /identity/v3/projects => generated 379 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:37.732871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:37.750395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc488074-8334-48d7-a3c3-4daee245bf31 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:37.812168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:37.814027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:37.931937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:37.934241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:37.934682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:37.936157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:37.937071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:37.940337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:37.963167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:37.996148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e2215-10e3-4337-b2fa-03ee04021667 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:38.003917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:38.005325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.005826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.006607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] PATH_INFO: `/v3/projects/f7e9ff3fdea14fa8afd51fddc3a8d0da` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.009230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e9ff3fdea14fa8afd51fddc3a8d0da)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:38.011863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:38.114477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 846/1686] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:37 2022] POST /identity/v3/users => generated 296 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:38.118302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:38.163011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82206560-dbf8-40bb-a696-e665018ae926 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:38.188923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e9ff3fdea14fa8afd51fddc3a8d0da', 'request_id': 'req-d94accf7-f5b7-40c7-9c15-37f78ff37749'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:38.311984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e9ff3fdea14fa8afd51fddc3a8d0da', 'request_id': 'req-d94accf7-f5b7-40c7-9c15-37f78ff37749'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:38.319838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e9ff3fdea14fa8afd51fddc3a8d0da', 'request_id': 'req-d94accf7-f5b7-40c7-9c15-37f78ff37749'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:38.350782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:38.352421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.354190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.354582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.355679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:38.361956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] The token cache is being invalidate because project f7e9ff3fdea14fa8afd51fddc3a8d0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:38.363903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:38.364332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:38.367782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d94accf7-f5b7-40c7-9c15-37f78ff37749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:38.375115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 841/1687] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:37 2022] DELETE /identity/v3/projects/f7e9ff3fdea14fa8afd51fddc3a8d0da => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:38.380348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe9ecb-3e0e-4000-bc13-5a04ba0f3fc1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.381211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe9ecb-3e0e-4000-bc13-5a04ba0f3fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.381386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe9ecb-3e0e-4000-bc13-5a04ba0f3fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.390295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 847/1688] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:06:38.394106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:38.407443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:38.422076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4cfe9ecb-3e0e-4000-bc13-5a04ba0f3fc1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:38.425008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4cfe9ecb-3e0e-4000-bc13-5a04ba0f3fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:38.427315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daee6e-2b1c-4436-8b44-95edd69687fe admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:38.439426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:38.440530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.441656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.442046 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] PATH_INFO: `/v3/projects/8f5beb1e152e44d0873f5e69fef8ebb8/users/cdc66a1460cc4620b2a8684d1442388e/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.457385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 842/1689] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:38.459190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.459615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.460644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.510913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5beb1e152e44d0873f5e69fef8ebb8, user_id=cdc66a1460cc4620b2a8684d1442388e, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:38.515242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:38.526065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:38.528347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:38.549641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 843/1690] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:38.553299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:38.573773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425ac2-eee1-4c02-a249-56be5b42c6e0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:38.600672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 848/1691] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:38 2022] PUT /identity/v3/projects/8f5beb1e152e44d0873f5e69fef8ebb8/users/cdc66a1460cc4620b2a8684d1442388e/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:38.602297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:38.604886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:38.610808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.611701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.612705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.619301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:38.619921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:38.623417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:38.669210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 844/1692] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:06:38.672249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:38.700554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c855d-7460-4477-80eb-6eee0687c251 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:38.800704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:38.806374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf856f4-7215-4619-b895-99fc6c9eb8b3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:38.834861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:38.835980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.836649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.839467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.840773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:38.844746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:38.885159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 849/1693] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:38.886993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:38.897578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:38.899624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:38.900507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:38.901484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:38.902424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:38.903154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:38.910407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:38.968551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc9e83-a656-47e7-ba88-bda9626f3e76 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:39.003731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:39.010179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.010179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.012876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] PATH_INFO: `/v3/users/5e7f13b5f714413396ead1663f9ffc50` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.019103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7f13b5f714413396ead1663f9ffc50)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:39.022880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:39.118449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 845/1694] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:38 2022] POST /identity/v3/projects => generated 389 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:39.124828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:39.323028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3984a6-cb8c-4e4b-be1e-1a57feb78857 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:39.323028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:39.323028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.323028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.323578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] PATH_INFO: `/v3/projects/8f5beb1e152e44d0873f5e69fef8ebb8/users/cdc66a1460cc4620b2a8684d1442388e/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.327913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5beb1e152e44d0873f5e69fef8ebb8, user_id=cdc66a1460cc4620b2a8684d1442388e, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:39.334502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:39.370086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 846/1695] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:39 2022] PUT /identity/v3/projects/8f5beb1e152e44d0873f5e69fef8ebb8/users/cdc66a1460cc4620b2a8684d1442388e/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:39.372823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:39.409601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7f13b5f714413396ead1663f9ffc50'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:39.410800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7f13b5f714413396ead1663f9ffc50'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:39.496894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7f13b5f714413396ead1663f9ffc50'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:39.509144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7f13b5f714413396ead1663f9ffc50'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:39.526696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:39.532266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e95d03-0c64-4cd2-af38-a7722a2c9941 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:39.543268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7fbca032-f43e-4abb-8e26-95ec8774a787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7f13b5f714413396ead1663f9ffc50'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:39.545939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfcd084-b9a3-49c0-8257-dfc724a91d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:39.547059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcd084-b9a3-49c0-8257-dfc724a91d43 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.547674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcd084-b9a3-49c0-8257-dfc724a91d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.548101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcd084-b9a3-49c0-8257-dfc724a91d43 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.551848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcd084-b9a3-49c0-8257-dfc724a91d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:39.554050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfcd084-b9a3-49c0-8257-dfc724a91d43 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:39.563018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 850/1696] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:38 2022] DELETE /identity/v3/users/5e7f13b5f714413396ead1663f9ffc50 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:39.568355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516f08-972c-4990-9149-f2f64c29b152 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.568663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516f08-972c-4990-9149-f2f64c29b152 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.568950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc516f08-972c-4990-9149-f2f64c29b152 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.572654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 851/1697] 10.222.0.58 () {54 vars in 960 bytes} [Thu Jun 16 18:06:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:06:39.578953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff8470-d7ef-4bb0-925c-83a6e0fd47a1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.579399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff8470-d7ef-4bb0-925c-83a6e0fd47a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.579694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff8470-d7ef-4bb0-925c-83a6e0fd47a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.610399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ffff8470-d7ef-4bb0-925c-83a6e0fd47a1 None None] MFA Rules not processed for user `70d6f15465b84182a28d96d1808a9660`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:39.612107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ffff8470-d7ef-4bb0-925c-83a6e0fd47a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:39.689768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 847/1698] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:39 2022] POST /identity/v3/users => generated 295 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:39.692546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.692999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.703952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.704270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 852/1699] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:39.707333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe3f2c-2cf6-4d1e-a52d-abbb971910ac None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.707863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe3f2c-2cf6-4d1e-a52d-abbb971910ac None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.719977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe3f2c-2cf6-4d1e-a52d-abbb971910ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.791348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1abe3f2c-2cf6-4d1e-a52d-abbb971910ac None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:39.792623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1abe3f2c-2cf6-4d1e-a52d-abbb971910ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:39.813021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] MFA Rules not processed for user `cdc66a1460cc4620b2a8684d1442388e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:39.822695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:39.942713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 853/1700] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:39.948419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:39.949080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:39.949504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:39.973393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:39.976060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:40.158651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 848/1701] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:40.158651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:40.158651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:40.158996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 854/1702] 10.222.0.58 () {58 vars in 1057 bytes} [Thu Jun 16 18:06:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:40.158996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:40.158996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:40.192036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707bae2-479a-4aeb-996d-681f23c3f6eb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:40.220910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:40.221987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.222484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.222956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:40.224268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:40.235212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:40.290685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 849/1703] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:40.292302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:40.313655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acdd33-990f-4ff1-99ff-f9b525309c0b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:40.323170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:40.324058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.324314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.326176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:40.335964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39873e-f8ec-4825-82a8-992d550b3f86 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:40.338447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:40.339904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:40.352254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 855/1704] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:40.353040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:40.354459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:40.356667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:40.358621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.359143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.359938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:40.369203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:40.417535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:40.421208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:40.478266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ab714-2d74-44f2-bfbc-597ecf47c784 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:40.483446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 850/1705] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:40.486426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:40.488239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:40.488923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.489219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.490184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:40.523176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:40.526881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:40.546190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc974c-e938-4e64-bd90-48dd4e1682a0 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:40.552090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 856/1706] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:40.593656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0ba59-a00c-4f2d-8f49-e78cac1dfcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:40.594702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ba59-a00c-4f2d-8f49-e78cac1dfcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.595132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ba59-a00c-4f2d-8f49-e78cac1dfcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.595599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ba59-a00c-4f2d-8f49-e78cac1dfcd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:40.604016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0ba59-a00c-4f2d-8f49-e78cac1dfcd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:40.606614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0ba59-a00c-4f2d-8f49-e78cac1dfcd5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:40.697836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 851/1707] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:40.704703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:40.720927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c32ab-87ca-4f52-ba6c-bad618973654 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:40.746584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:40.746584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.746584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.746584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] PATH_INFO: `/v3/projects/f7273833e0de4babb7ec041abf85ba41/users/d21eada1147b447e9a0cd75346d26509/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:40.769562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:40.786006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7273833e0de4babb7ec041abf85ba41, user_id=d21eada1147b447e9a0cd75346d26509, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:40.790527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:40.914314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:40.920852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 857/1708] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:40 2022] PUT /identity/v3/projects/f7273833e0de4babb7ec041abf85ba41/users/d21eada1147b447e9a0cd75346d26509/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:06:40.942056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:40.942776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:40.942776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:41.035739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] MFA Rules not processed for user `d21eada1147b447e9a0cd75346d26509`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:41.041046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:41.166494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 852/1709] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:41.173772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:41.518947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b27e5-e121-4e16-a5db-d9070436be19 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:41.547750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e596eb-357e-4728-9812-bb7aef20b46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:41.550035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e596eb-357e-4728-9812-bb7aef20b46e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:41.550525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e596eb-357e-4728-9812-bb7aef20b46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:41.552122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e596eb-357e-4728-9812-bb7aef20b46e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:41.554247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e596eb-357e-4728-9812-bb7aef20b46e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:41.562016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e596eb-357e-4728-9812-bb7aef20b46e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:41.811619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 858/1710] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:41 2022] POST /identity/v3/projects => generated 387 bytes in 639 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:06:41.812409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:41.829106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63c8d8-5d64-4fa0-a304-31e696fb5325 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:41.929052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcc292-1c3f-41b7-b861-3f23befbfd16 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:41.934305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcc292-1c3f-41b7-b861-3f23befbfd16 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:41.934305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcc292-1c3f-41b7-b861-3f23befbfd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:41.987956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:42.009696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:42.010708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:42.012012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:42.012601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:42.017234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:42.021344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:42.054435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-26bcc292-1c3f-41b7-b861-3f23befbfd16 None None] MFA Rules not processed for user `9eac1fb288764b74baf30892237b3cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:42.057252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-26bcc292-1c3f-41b7-b861-3f23befbfd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:42.155063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 853/1711] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:41 2022] POST /identity/v3/users => generated 300 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:42.172327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:42.188796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5725b-9f19-4d5c-bd58-e6313f89a8b0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:42.196718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:42.199574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:42.200033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:42.200608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:42.201611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:42.204982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:42.229736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 859/1712] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:42.250337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:42.256031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:42.267604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:42.268334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:42.275820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 854/1713] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:42.281757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:42.331838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f49d3-677f-4bad-af1d-5ef92d6f67d2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:42.334042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] MFA Rules not processed for user `9eac1fb288764b74baf30892237b3cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:42.335301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:42.346661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:42.349665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:42.349966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:42.352230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] PATH_INFO: `/v3/projects/4945a4c4bed141709b5395021bdf16d9/users/d8197130c09b4343abfb71ecfd433b28/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:42.360323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 860/1714] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:42.372901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:42.374904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:42.389998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4945a4c4bed141709b5395021bdf16d9, user_id=d8197130c09b4343abfb71ecfd433b28, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:42.412733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:42.659882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 855/1715] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:42 2022] PUT /identity/v3/projects/4945a4c4bed141709b5395021bdf16d9/users/d8197130c09b4343abfb71ecfd433b28/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:42.660040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5108f-fcbb-464f-8cf5-e4df381227ff None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:42.661431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:42.824111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:42.826354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:42.827459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:42.827919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:42.890995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb33cc4-c293-4750-bd89-d6a7916cde14 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:42.904287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdc66a1460cc4620b2a8684d1442388e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5beb1e152e44d0873f5e69fef8ebb8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:42.907923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:42.908262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:42.908550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:42.909336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:42.909661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:42.911333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:42.944630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 856/1716] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:42.954100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:42.957940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:42.961138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:43.006062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 861/1717] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:43.007145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:43.013949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:43.145040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:43.149198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35471-3617-4947-a0ab-632f27dded86 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:43.161859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:43.162708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.163126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.163544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.164777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:43.170898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:43.209771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 857/1718] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:43.210847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:43.232543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9d7d4-23f1-40f3-8d1c-a749268d4871 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:43.248950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242b7a2-b684-423a-99e9-6e4334fbcec0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:43.263964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:43.269793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.270132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.270424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] PATH_INFO: `/v3/projects/ac7543e92c9b46c2b836bc5c33c43f40` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.270931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdc66a1460cc4620b2a8684d1442388e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5beb1e152e44d0873f5e69fef8ebb8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:43.271776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.272179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.272562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.274064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7543e92c9b46c2b836bc5c33c43f40)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:43.275091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:43.281027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:43.285840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:43.460040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 858/1719] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:43 2022] POST /identity/v3/projects => generated 353 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:43.461732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:43.524426 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7543e92c9b46c2b836bc5c33c43f40', 'request_id': 'req-e2ce1366-6ec3-4bcc-a702-351509bb1141'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:43.550526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7543e92c9b46c2b836bc5c33c43f40', 'request_id': 'req-e2ce1366-6ec3-4bcc-a702-351509bb1141'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:43.579888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7543e92c9b46c2b836bc5c33c43f40', 'request_id': 'req-e2ce1366-6ec3-4bcc-a702-351509bb1141'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:43.677241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] The token cache is being invalidate because project ac7543e92c9b46c2b836bc5c33c43f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:43.677602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:43.678338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:43.680709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 862/1720] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:43 2022] DELETE /identity/v3/projects/ac7543e92c9b46c2b836bc5c33c43f40 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:43.684492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:43.695692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:43.703755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53d50-c9b0-43f4-a7b9-b5054f509d71 tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:43.870504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce1366-6ec3-4bcc-a702-351509bb1141 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:43.876830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdc66a1460cc4620b2a8684d1442388e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5beb1e152e44d0873f5e69fef8ebb8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:43.877711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.878003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.878288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.878542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662cddc7-0a14-43b3-a555-e648961cdd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:43.879236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:43.879487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cddc7-0a14-43b3-a555-e648961cdd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.879613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:43.880132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cddc7-0a14-43b3-a555-e648961cdd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.880421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662cddc7-0a14-43b3-a555-e648961cdd25 admin admin] PATH_INFO: `/v3/projects/4945a4c4bed141709b5395021bdf16d9/users/d8197130c09b4343abfb71ecfd433b28/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.881175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277a148-2529-4316-9c16-45ba8887538a tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:43.886108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cddc7-0a14-43b3-a555-e648961cdd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4945a4c4bed141709b5395021bdf16d9, user_id=d8197130c09b4343abfb71ecfd433b28, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:43.889738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662cddc7-0a14-43b3-a555-e648961cdd25 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:43.898102 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 863/1721] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:43.900028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.904391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.904391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.929807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 859/1722] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:43 2022] PUT /identity/v3/projects/4945a4c4bed141709b5395021bdf16d9/users/d8197130c09b4343abfb71ecfd433b28/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:43.932445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26751b-f908-4af1-9353-758a18c25e4b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:43.932939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26751b-f908-4af1-9353-758a18c25e4b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:43.933383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26751b-f908-4af1-9353-758a18c25e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:43.988330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:43.992204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:44.006038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cb26751b-f908-4af1-9353-758a18c25e4b None None] MFA Rules not processed for user `30ac12728e8f4ec9af534612030d75eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:44.008209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cb26751b-f908-4af1-9353-758a18c25e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:44.197950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 860/1723] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:44.199883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:44.201043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:44.201650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:44.234459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 864/1724] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:44.239897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:44.312451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] MFA Rules not processed for user `d8197130c09b4343abfb71ecfd433b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:44.317697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:44.427731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbd95b-e520-40a6-8f82-224265054c92 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:44.435343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdc66a1460cc4620b2a8684d1442388e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8f5beb1e152e44d0873f5e69fef8ebb8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:44.437597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:44.439079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:44.440821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] PATH_INFO: `/v3/projects/d9fa118da41148b08f8f55249c1fb20a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:44.445012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d9fa118da41148b08f8f55249c1fb20a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:44.454747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:44.484648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 861/1725] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:44.485722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:44.488219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:44.636109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac397da-6a5a-4b07-9315-8cc1421f354d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:44.642794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78042f40-ab26-4da0-a4ec-b551a19fe490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:44.643500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78042f40-ab26-4da0-a4ec-b551a19fe490 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:44.643786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78042f40-ab26-4da0-a4ec-b551a19fe490 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:44.644065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78042f40-ab26-4da0-a4ec-b551a19fe490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:44.659496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78042f40-ab26-4da0-a4ec-b551a19fe490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:44.661092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78042f40-ab26-4da0-a4ec-b551a19fe490 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:44.687035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 862/1726] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:44.692115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:44.692367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:44.693129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:44.741840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:44.745192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:44.768112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9fa118da41148b08f8f55249c1fb20a', 'request_id': 'req-f4d1c372-3eee-4342-a888-282eb2ade89e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:44.786920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9fa118da41148b08f8f55249c1fb20a', 'request_id': 'req-f4d1c372-3eee-4342-a888-282eb2ade89e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:44.793482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9fa118da41148b08f8f55249c1fb20a', 'request_id': 'req-f4d1c372-3eee-4342-a888-282eb2ade89e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:44.924888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] The token cache is being invalidate because project d9fa118da41148b08f8f55249c1fb20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:44.924888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:44.924888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:44.927219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 865/1727] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:44 2022] DELETE /identity/v3/projects/d9fa118da41148b08f8f55249c1fb20a => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:44.928273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:44.934833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:45.037029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 863/1728] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:45.038358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:45.053154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819b9a4-7667-4ead-975e-b8decd513274 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:45.056195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:45.078882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:45.079604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:45.079896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:45.080332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:45.081408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:45.081729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:45.087779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:45.125900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1c372-3eee-4342-a888-282eb2ade89e tempest-VolumeQuotaClassesTest-833794470 tempest-VolumeQuotaClassesTest-833794470-project-admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:45.137640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:45.138584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:45.139729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:45.140655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:45.141932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 864/1729] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:45.142965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:45.148350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:45.161960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e36d-aadc-41e9-9dac-0b36a700dfda admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:45.172417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:45.173103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:45.175315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:45.175807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:45.177759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:45.181889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:45.416262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:45.425192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:45.428780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:45.446130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 866/1730] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:45.545626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:45.546142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 865/1731] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:45 2022] POST /identity/v3/projects => generated 357 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:45.568980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53308-facb-4068-b957-15d9d96984bf service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:45.847457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:45.850526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:45.850526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:45.850526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:45.853975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:45.869291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:45.872789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:45.876711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:45.934119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de32ae-b651-4c3c-9eea-2c368ccc0a2c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:45.971834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:45.977430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:45.978878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:45.979608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] PATH_INFO: `/v3/users/d21eada1147b447e9a0cd75346d26509` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:45.993822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21eada1147b447e9a0cd75346d26509)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:45.993822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:46.066448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 867/1732] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:45 2022] POST /identity/v3/users => generated 279 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:46.074359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:46.114131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353beff-6fdc-4fe2-a0cb-8e4f64901dad admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:46.137185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:46.139327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:46.143544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:46.144283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:46.149976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:46.152061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:46.194345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 868/1733] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:06:46.201705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:46.229036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a63127-345b-4100-85ab-fc313277cf13 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:46.245238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:46.248834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:46.248834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:46.248834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:46.252086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:46.253868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:46.306561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 869/1734] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:46.313292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:46.348381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b63f-0255-494b-b286-f5d5f39f5ada admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:46.364734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:46.365954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:46.366330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:46.366738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] PATH_INFO: `/v3/projects/f091a0b898984656b546e3609f567631/users/9b1249cd79404d8ba6ba80fb8ac38535/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:46.423869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f091a0b898984656b546e3609f567631, user_id=9b1249cd79404d8ba6ba80fb8ac38535, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:46.426648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:46.458884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21eada1147b447e9a0cd75346d26509'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:46.504251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21eada1147b447e9a0cd75346d26509'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:46.525654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21eada1147b447e9a0cd75346d26509'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:46.562120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 870/1735] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:46 2022] PUT /identity/v3/projects/f091a0b898984656b546e3609f567631/users/9b1249cd79404d8ba6ba80fb8ac38535/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:46.565080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:46.570132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:46.761188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21eada1147b447e9a0cd75346d26509'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:46.761188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7170ac1a-0578-4594-8ffa-766f84dd472c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21eada1147b447e9a0cd75346d26509'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:46.770906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b1ff1-0fec-45a8-bfad-456f56178dce admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:46.771849 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 866/1736] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:45 2022] DELETE /identity/v3/users/d21eada1147b447e9a0cd75346d26509 => generated 0 bytes in 898 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:46.773300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9887-4657-4502-b459-ea1d1eb9ebf1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:46.773557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9887-4657-4502-b459-ea1d1eb9ebf1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:46.773839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a9887-4657-4502-b459-ea1d1eb9ebf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:46.849178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9d4a9887-4657-4502-b459-ea1d1eb9ebf1 None None] MFA Rules not processed for user `712089cbbf9a424fa5d4412ced2cdc58`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:46.849719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9d4a9887-4657-4502-b459-ea1d1eb9ebf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:46.936586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:46.937498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:46.937790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:46.938499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] PATH_INFO: `/v3/users/72c29f36e9614fb4905fdfd34b1859ba` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:46.947207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c29f36e9614fb4905fdfd34b1859ba)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:46.952977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:46.968851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:47.026043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 867/1737] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:47.028284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf0fe8-597e-4960-8abd-6b9ce6c63045 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:47.028284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf0fe8-597e-4960-8abd-6b9ce6c63045 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:47.028731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf0fe8-597e-4960-8abd-6b9ce6c63045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:47.082987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fcaf0fe8-597e-4960-8abd-6b9ce6c63045 None None] MFA Rules not processed for user `9b1249cd79404d8ba6ba80fb8ac38535`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:47.084629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fcaf0fe8-597e-4960-8abd-6b9ce6c63045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.186548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 868/1738] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:47.188351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a43d5-81f5-49fc-8bab-b37d4182905d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:47.188863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a43d5-81f5-49fc-8bab-b37d4182905d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:47.189613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a43d5-81f5-49fc-8bab-b37d4182905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:47.209788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5d0a43d5-81f5-49fc-8bab-b37d4182905d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:47.211105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5d0a43d5-81f5-49fc-8bab-b37d4182905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.220759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 869/1739] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:47.222881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ab559-65ec-4c1e-9d36-6f00996eba8b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:47.237835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ab559-65ec-4c1e-9d36-6f00996eba8b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:47.238164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ab559-65ec-4c1e-9d36-6f00996eba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:47.280860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c33ab559-65ec-4c1e-9d36-6f00996eba8b None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:47.282121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c33ab559-65ec-4c1e-9d36-6f00996eba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.348675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:47.359898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c29f36e9614fb4905fdfd34b1859ba'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:47.360703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c29f36e9614fb4905fdfd34b1859ba'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:47.436549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 870/1740] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:47.438564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:47.438864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:47.439139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:47.476752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] MFA Rules not processed for user `8498989e673d445dbc70aa9df55f391b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:47.478001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.502767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c29f36e9614fb4905fdfd34b1859ba'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:47.540848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c29f36e9614fb4905fdfd34b1859ba'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:47.588131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c29f36e9614fb4905fdfd34b1859ba'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:47.626018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 871/1741] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:46 2022] DELETE /identity/v3/users/72c29f36e9614fb4905fdfd34b1859ba => generated 0 bytes in 1060 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:47.626018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:47.638485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.754764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 871/1742] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:47.758889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:47.768213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.812281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a60629-5014-46da-a8bd-6dfa13bec6ef admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:47.820612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fc076-dc00-4f67-bb60-c9d58be34ba8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:47.821376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:47.822117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:47.822449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:47.822740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:47.825508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:47.977142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:47.979208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:47.996438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 872/1743] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:47.997909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:47.999482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:48.026545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21190b2-5249-451d-9c5c-ad320d15408a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.027323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21190b2-5249-451d-9c5c-ad320d15408a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.029343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21190b2-5249-451d-9c5c-ad320d15408a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.030364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21190b2-5249-451d-9c5c-ad320d15408a admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.032410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21190b2-5249-451d-9c5c-ad320d15408a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.036793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21190b2-5249-451d-9c5c-ad320d15408a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.133240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a9ef1-0ba9-4890-85c9-566e1ef1e43a service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.140348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.141042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.141328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.141610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.157984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:48.161364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 872/1744] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:47 2022] POST /identity/v3/projects => generated 357 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:48.166101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.166573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.167016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.185742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.189160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.194875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:48.196203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:48.206920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 873/1745] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:48.207947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:48.209378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:48.229496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679e5fc-b9ef-417f-bef1-6f8f4f9b8ac2 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.298489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 873/1746] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:48.302933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:48.307558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:48.395980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:48.399796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.401722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.402026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.402315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.439046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8e9d6-4e20-45d0-bac6-6a1d3d782cc0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.452430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.453079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.453370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.453653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.458262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:48.512013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.514715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.528188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 874/1747] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:48.531358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:48.542767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f17d03-adc9-4693-95ad-d8e42f6b16c7 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.548929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.549823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.550125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.550424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.551125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.553664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.618709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.625201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.648860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 874/1748] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:06:48.648860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:48.715014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 875/1749] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:48 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:48.716579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:48.748970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df48b84-1f35-4740-81da-2dc52fcff00c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.792373 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.794931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.794931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.794931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.805173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff461d-05f2-4f35-8658-b67d35c83ef6 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.807527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.809333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.823165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.824564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.825396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.825953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.838338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.842234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:48.877271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 875/1750] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:48.882299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:48.883511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 876/1751] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:48.906490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d29f9-9fe9-487f-8907-e6742b3121f2 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:48.923838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:48.926379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:48.929080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:48.930093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] PATH_INFO: `/v3/projects/08f35610c52c423d80093b7dfbb56640/users/85549e8ce42d4425b618227ff51654c7/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:48.984357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f35610c52c423d80093b7dfbb56640, user_id=85549e8ce42d4425b618227ff51654c7, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:48.984357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:49.187246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 876/1752] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:48 2022] PUT /identity/v3/projects/08f35610c52c423d80093b7dfbb56640/users/85549e8ce42d4425b618227ff51654c7/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:49.191257 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:49.213560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0655cbe-e14e-4b4d-a557-31fa3496ea66 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:49.314497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:49.331884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7a787-d6a2-4a26-aa12-efdc3da2125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:49.332682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7a787-d6a2-4a26-aa12-efdc3da2125e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:49.334117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7a787-d6a2-4a26-aa12-efdc3da2125e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:49.334490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7a787-d6a2-4a26-aa12-efdc3da2125e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:49.335249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7a787-d6a2-4a26-aa12-efdc3da2125e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:49.336933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7a787-d6a2-4a26-aa12-efdc3da2125e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:49.367044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 877/1753] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:49.373182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:49.393432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721716-9c57-48d1-b1f9-ee3235bfda53 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:49.416668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:49.418719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:49.418719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:49.418926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] PATH_INFO: `/v3/projects/08f35610c52c423d80093b7dfbb56640/users/85549e8ce42d4425b618227ff51654c7/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:49.443932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f35610c52c423d80093b7dfbb56640, user_id=85549e8ce42d4425b618227ff51654c7, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:49.463184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:49.515542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 877/1754] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:49 2022] PUT /identity/v3/projects/08f35610c52c423d80093b7dfbb56640/users/85549e8ce42d4425b618227ff51654c7/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:49.519654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dd8d-a426-44a7-96db-eec1134ce167 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:49.520214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dd8d-a426-44a7-96db-eec1134ce167 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:49.520214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438dd8d-a426-44a7-96db-eec1134ce167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:49.566983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4438dd8d-a426-44a7-96db-eec1134ce167 None None] MFA Rules not processed for user `85549e8ce42d4425b618227ff51654c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:49.568294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4438dd8d-a426-44a7-96db-eec1134ce167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:49.707147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 878/1755] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:49.715529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:49.717718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:49.801983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:49.923408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fdc27-e5bd-4620-9810-642394f0fdf2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:49.937633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:49.938013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:49.938408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:49.938791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:49.941192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:49.941806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:49.943087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:49.977245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:49.982301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:49.997182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] MFA Rules not processed for user `6aae965af8284363820254c9286adf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:49.999197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:50.036684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 878/1756] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:49 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:50.215201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 879/1757] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jun 16 18:06:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:50.263216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:50.272997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:50.305801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7548fd-a335-4525-85ca-00023f5d42ac service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:50.321551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:50.322681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:50.323335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:50.333128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:50.360014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:50.363912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:50.396507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 879/1758] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:50.454057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:50.473579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2b5b4-ea23-4a08-b10b-17aea4871a84 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:50.494013 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:50.515955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:50.516459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:50.516752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:50.517042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:50.546802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:50.719105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:50.720641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:50.745629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 880/1759] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:50 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:51.384822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:52.284006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:52.289921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:52.407685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:52.545398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd3bb7-1eef-4497-9e4d-a4e9a66a152c service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:52.557473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:52.560296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:52.560645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:52.560910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] PATH_INFO: `/v3/projects/e8a415c2319f4a57abc14cb56e40ecef` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:52.566924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a415c2319f4a57abc14cb56e40ecef)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:52.571929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:52.870207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a415c2319f4a57abc14cb56e40ecef', 'request_id': 'req-557b7861-3893-49ca-a75a-f2014efe5eec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:52.948145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:52.953613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:52.973348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a415c2319f4a57abc14cb56e40ecef', 'request_id': 'req-557b7861-3893-49ca-a75a-f2014efe5eec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:52.984415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a415c2319f4a57abc14cb56e40ecef', 'request_id': 'req-557b7861-3893-49ca-a75a-f2014efe5eec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.016725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c4d27-9236-43d1-93fa-d900e91f3021 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:53.071455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] The token cache is being invalidate because project e8a415c2319f4a57abc14cb56e40ecef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:53.073753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.076100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.097676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 880/1760] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:52 2022] DELETE /identity/v3/projects/e8a415c2319f4a57abc14cb56e40ecef => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:06:53.100129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:53.106077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:53.205182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:53.357162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:53.358166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:53.364259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:53.366967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] PATH_INFO: `/v3/projects/f7273833e0de4babb7ec041abf85ba41` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:53.374156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7273833e0de4babb7ec041abf85ba41)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:53.384837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:53.423113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b7861-3893-49ca-a75a-f2014efe5eec admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:53.453525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:53.454446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:53.454737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:53.455129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] PATH_INFO: `/v3/users/cdc66a1460cc4620b2a8684d1442388e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:53.484021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc66a1460cc4620b2a8684d1442388e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:53.484021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:53.677952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7273833e0de4babb7ec041abf85ba41', 'request_id': 'req-aa9db80a-140e-44c4-bdba-57bedee92e53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.692298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7273833e0de4babb7ec041abf85ba41', 'request_id': 'req-aa9db80a-140e-44c4-bdba-57bedee92e53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.707274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7273833e0de4babb7ec041abf85ba41', 'request_id': 'req-aa9db80a-140e-44c4-bdba-57bedee92e53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.718146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:53.841673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] The token cache is being invalidate because project f7273833e0de4babb7ec041abf85ba41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:53.847315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.847315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:53.847315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 881/1761] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:52 2022] DELETE /identity/v3/projects/f7273833e0de4babb7ec041abf85ba41 => generated 0 bytes in 900 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:06:53.851948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:53.857682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:54.038383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc66a1460cc4620b2a8684d1442388e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:54.038383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc66a1460cc4620b2a8684d1442388e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:54.038383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc66a1460cc4620b2a8684d1442388e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:54.239177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db80a-140e-44c4-bdba-57bedee92e53 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:54.269648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc66a1460cc4620b2a8684d1442388e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:54.274557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89c02083-d979-4979-8ce9-2b1b3aed529c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc66a1460cc4620b2a8684d1442388e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:54.283324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 881/1762] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:53 2022] DELETE /identity/v3/users/cdc66a1460cc4620b2a8684d1442388e => generated 0 bytes in 1181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:54.284704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:54.286645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:54.288978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:54.289652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] PATH_INFO: `/v3/users/d8197130c09b4343abfb71ecfd433b28` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:54.292493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8197130c09b4343abfb71ecfd433b28)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:54.298880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:54.912670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8197130c09b4343abfb71ecfd433b28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:54.923407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8197130c09b4343abfb71ecfd433b28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:55.029090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8197130c09b4343abfb71ecfd433b28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:55.098671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8197130c09b4343abfb71ecfd433b28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:55.102674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6e70caa4-0a80-46b3-ac17-15cbf7e29d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8197130c09b4343abfb71ecfd433b28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:55.126161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 882/1763] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:06:53 2022] DELETE /identity/v3/users/d8197130c09b4343abfb71ecfd433b28 => generated 0 bytes in 1271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:56.035622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:06:56.651865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508907-a4d8-49bf-a942-1982089c5d9d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:56.653137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508907-a4d8-49bf-a942-1982089c5d9d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:56.653827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508907-a4d8-49bf-a942-1982089c5d9d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:56.656441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 882/1764] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:06:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:06:56.706569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:56.707521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:56.708035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:56.765880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:56.773567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:56.945445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 883/1765] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:06:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:57.033804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508907-a4d8-49bf-a942-1982089c5d9d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:57.035843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f5508907-a4d8-49bf-a942-1982089c5d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:57.071143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508907-a4d8-49bf-a942-1982089c5d9d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:57.079635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f97f60-f7b9-4902-9164-e5961b70c061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:57.080640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f97f60-f7b9-4902-9164-e5961b70c061 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:57.081089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f97f60-f7b9-4902-9164-e5961b70c061 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:57.081477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f97f60-f7b9-4902-9164-e5961b70c061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:57.111444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f97f60-f7b9-4902-9164-e5961b70c061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:57.115388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f97f60-f7b9-4902-9164-e5961b70c061 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:57.177333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 883/1766] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:06:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:06:57.673059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:57.674889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:57.818710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa158075-ddf8-4077-95a4-0e2674a007ad None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:57.831342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:57.833692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:57.835380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:57.836628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] PATH_INFO: `/v3/projects/8f5beb1e152e44d0873f5e69fef8ebb8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:57.839531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5beb1e152e44d0873f5e69fef8ebb8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:57.843843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:58.025503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5beb1e152e44d0873f5e69fef8ebb8', 'request_id': 'req-cf224d45-874c-4ac6-8b7b-e462856c6047'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:58.042231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5beb1e152e44d0873f5e69fef8ebb8', 'request_id': 'req-cf224d45-874c-4ac6-8b7b-e462856c6047'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:58.071681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5beb1e152e44d0873f5e69fef8ebb8', 'request_id': 'req-cf224d45-874c-4ac6-8b7b-e462856c6047'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:58.302161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] The token cache is being invalidate because project 8f5beb1e152e44d0873f5e69fef8ebb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:06:58.315419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:58.315419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cf224d45-874c-4ac6-8b7b-e462856c6047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:06:58.315419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 884/1767] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:57 2022] DELETE /identity/v3/projects/8f5beb1e152e44d0873f5e69fef8ebb8 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:06:58.316354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712c647-45b3-40a6-9ac9-191ed6e271d7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:58.316869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712c647-45b3-40a6-9ac9-191ed6e271d7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:58.317005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712c647-45b3-40a6-9ac9-191ed6e271d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:58.354298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c712c647-45b3-40a6-9ac9-191ed6e271d7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:58.359827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c712c647-45b3-40a6-9ac9-191ed6e271d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:58.507762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:58.508757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:58.509724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:58.527035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 884/1768] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:06:58.537278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:58.537662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:58.538195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:58.574234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] MFA Rules not processed for user `30ac12728e8f4ec9af534612030d75eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:58.577729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:58.578355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:06:58.580043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:06:58.596437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 885/1769] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:58.600837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:58.618086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c826b6-c323-4913-b42a-0e57d954f89c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:58.626065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:58.627154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:58.627841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:58.628351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:58.629516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:06:58.630941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:58.635964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:58.655583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 886/1770] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:06:58.659709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:58.682549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576125-031a-441a-b3a7-2d70c820e43a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:58.700480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:58.702204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:58.703646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:58.706202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:58.706985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:58.725515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:58.726465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 885/1771] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:06:59.160572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 887/1772] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:06:58 2022] POST /identity/v3/projects => generated 385 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:59.161730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:59.186808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6ea98-036c-42a7-a21d-7325d4001ac4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:59.443314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:59.446981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:59.446981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:59.446981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:59.446981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:59.451258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:59.632190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 886/1773] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:06:59 2022] POST /identity/v3/users => generated 293 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:06:59.633362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:59.635860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda657-251b-4c44-95c5-2623b93ef0f4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:59.646426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:59.652635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:59.652635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:59.652635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:59.652635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:59.652635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:59.715801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:59.725045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 888/1774] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:06:59.744571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd0a75-6381-4fe2-b418-992cdcf7c319 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:59.769938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:59.770738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:59.771035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:59.771295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:59.772075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:06:59.781652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:06:59.836535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:59.847772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 887/1775] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:06:59.861441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4e678-4b74-4057-ad87-7c0534a0bd1d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:06:59.881045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7812f01-7da7-46c5-9291-0274d89cbdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:06:59.884564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7812f01-7da7-46c5-9291-0274d89cbdfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:06:59.884823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7812f01-7da7-46c5-9291-0274d89cbdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:06:59.887917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7812f01-7da7-46c5-9291-0274d89cbdfa admin admin] PATH_INFO: `/v3/projects/5e2f354bcbe04a469e80e87fa12e8ca2/users/560282dbe22b4a3d8195694b7842a7d1/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:06:59.963871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:06:59.968023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:00.046760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7812f01-7da7-46c5-9291-0274d89cbdfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2f354bcbe04a469e80e87fa12e8ca2, user_id=560282dbe22b4a3d8195694b7842a7d1, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:00.063552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7758bb-f420-48f8-83d7-3d3e6b940b6d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:00.068523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7812f01-7da7-46c5-9291-0274d89cbdfa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:00.086675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:00.091055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.091699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.092178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] PATH_INFO: `/v3/projects/4945a4c4bed141709b5395021bdf16d9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.094196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4945a4c4bed141709b5395021bdf16d9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:00.096478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:00.226770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 889/1776] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:06:59 2022] PUT /identity/v3/projects/5e2f354bcbe04a469e80e87fa12e8ca2/users/560282dbe22b4a3d8195694b7842a7d1/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:00.245017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce421a2-b407-4aeb-a3c5-b43cd23760e4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.247486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce421a2-b407-4aeb-a3c5-b43cd23760e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.251314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce421a2-b407-4aeb-a3c5-b43cd23760e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.320942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4945a4c4bed141709b5395021bdf16d9', 'request_id': 'req-c6c60acc-543e-4987-990b-cb50f37b0662'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:00.338236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4945a4c4bed141709b5395021bdf16d9', 'request_id': 'req-c6c60acc-543e-4987-990b-cb50f37b0662'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:00.361166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4945a4c4bed141709b5395021bdf16d9', 'request_id': 'req-c6c60acc-543e-4987-990b-cb50f37b0662'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:00.365518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dce421a2-b407-4aeb-a3c5-b43cd23760e4 None None] MFA Rules not processed for user `560282dbe22b4a3d8195694b7842a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:00.368168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dce421a2-b407-4aeb-a3c5-b43cd23760e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:00.578585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 890/1777] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:00.594873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b3de8-2673-48b2-b366-dfb9bd9848ba None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.595260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b3de8-2673-48b2-b366-dfb9bd9848ba None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.596095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] The token cache is being invalidate because project 4945a4c4bed141709b5395021bdf16d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:00.596228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b3de8-2673-48b2-b366-dfb9bd9848ba None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.596649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:00.597624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c6c60acc-543e-4987-990b-cb50f37b0662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:00.598295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 891/1778] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:07:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:07:00.599454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b3de8-2673-48b2-b366-dfb9bd9848ba None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:00.601553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-993b3de8-2673-48b2-b366-dfb9bd9848ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:00.602383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 888/1779] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:06:59 2022] DELETE /identity/v3/projects/4945a4c4bed141709b5395021bdf16d9 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:00.605566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:00.609268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149837f-dfb6-4c37-8980-19d6136c0912 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.609565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149837f-dfb6-4c37-8980-19d6136c0912 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.609878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149837f-dfb6-4c37-8980-19d6136c0912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.675810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2149837f-dfb6-4c37-8980-19d6136c0912 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:00.680166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2149837f-dfb6-4c37-8980-19d6136c0912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:00.767078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b3de8-2673-48b2-b366-dfb9bd9848ba None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:00.783509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:00.784168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.784562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.784998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.793241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:00.850392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:00.855617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283c981-d7f3-4cb2-bada-c726a56febe9 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:00.893649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 889/1780] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:00.895528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba293d-3ee8-493e-ad00-50f3a577682f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.897973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba293d-3ee8-493e-ad00-50f3a577682f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.898568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba293d-3ee8-493e-ad00-50f3a577682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.955879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 892/1781] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:00.959180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:00.960061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:00.961881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:00.999030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7dba293d-3ee8-493e-ad00-50f3a577682f None None] MFA Rules not processed for user `8498989e673d445dbc70aa9df55f391b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:01.006800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7dba293d-3ee8-493e-ad00-50f3a577682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:01.049904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:01.052901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:01.184605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 890/1782] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:01.187838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:01.188651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:01.198305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:01.255494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:01.262594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:01.267539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 893/1783] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:07:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:01.307466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:01.317924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 891/1784] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:01.325305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:01.358232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4ae50-3f2d-4166-a83e-17ff2bd76ef3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:01.372661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:01.373794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:01.374297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:01.374789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:01.376048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:01.378426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:01.383855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:01.423171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 894/1785] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:01.427752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:01.465122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58949f8-a068-4fc9-8798-c0839c535628 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:01.475709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f8fd4-aafe-49fe-9ea7-09c0c4abe4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:01.477106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8fd4-aafe-49fe-9ea7-09c0c4abe4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:01.478202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8fd4-aafe-49fe-9ea7-09c0c4abe4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:01.479240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f8fd4-aafe-49fe-9ea7-09c0c4abe4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:01.481169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f8fd4-aafe-49fe-9ea7-09c0c4abe4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:01.484040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f8fd4-aafe-49fe-9ea7-09c0c4abe4f4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:01.662567 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 892/1786] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:01 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:01.672856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:01.712590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b1ad-bd95-4a55-a1c6-cf6486c751ac admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:01.742356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:01.742654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:01.742938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:01.783854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] MFA Rules not processed for user `712089cbbf9a424fa5d4412ced2cdc58`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:01.785120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:01.885968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:01.887239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:01.897867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:01.898418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:01.899426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:01.902718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:01.940651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 893/1787] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:01.940651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:01.940651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:02.004154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 895/1788] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:01 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:02.005280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:02.051995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a093867-f492-4428-95fe-74b9c46f4f93 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:02.067933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:02.067933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:02.067933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:02.067933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:02.068294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:02.068294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:02.116020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 896/1789] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:02.125930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:02.142680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9be21-11e4-4d69-84a2-8975375d0ea1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:02.155336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:02.156114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:02.156407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:02.156965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:02.157711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:02.159544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:02.169561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af8230-276d-42a7-9e4c-09e6b94e6a5a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:02.178954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c7d72-bcb0-4b45-bffb-c010babf6d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:02.179881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7d72-bcb0-4b45-bffb-c010babf6d65 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:02.180175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7d72-bcb0-4b45-bffb-c010babf6d65 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:02.180506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c7d72-bcb0-4b45-bffb-c010babf6d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:02.191163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 897/1790] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:02.192823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:02.207649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c7d72-bcb0-4b45-bffb-c010babf6d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:02.212045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c7d72-bcb0-4b45-bffb-c010babf6d65 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:02.223703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857280-8c48-482d-80e7-a6db22b42d41 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:02.232045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 894/1791] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:02.237313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:02.238311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:02.238662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:02.239086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] PATH_INFO: `/v3/projects/c528667c6413410b8d3abfb4c8abb4d8/users/12d7fe122d5742e39422ada4aaabfd10/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:02.281875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c528667c6413410b8d3abfb4c8abb4d8, user_id=12d7fe122d5742e39422ada4aaabfd10, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:02.287536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:02.613119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 898/1792] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:02 2022] PUT /identity/v3/projects/c528667c6413410b8d3abfb4c8abb4d8/users/12d7fe122d5742e39422ada4aaabfd10/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:02.624885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacc324-5e65-4406-9e70-723c077bf706 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:02.625763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacc324-5e65-4406-9e70-723c077bf706 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:02.626212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bacc324-5e65-4406-9e70-723c077bf706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:02.709905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3bacc324-5e65-4406-9e70-723c077bf706 None None] MFA Rules not processed for user `12d7fe122d5742e39422ada4aaabfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:02.709905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3bacc324-5e65-4406-9e70-723c077bf706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:02.915291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 895/1793] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:02.916499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:02.968166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79201d-1839-46f8-8b67-d321ba468946 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:02.972629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79201d-1839-46f8-8b67-d321ba468946 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:02.973001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79201d-1839-46f8-8b67-d321ba468946 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:02.975844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 896/1794] 10.222.0.58 () {54 vars in 912 bytes} [Thu Jun 16 18:07:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:07:03.003982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:03.003982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:03.003982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:03.041068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] MFA Rules not processed for user `6aae965af8284363820254c9286adf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:03.044039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:03.129807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897651ab-3920-415c-b6ca-492535d767a1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:03.158375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:03.159072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:03.163858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:03.164153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:03.169089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:03.170855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 897/1795] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:07:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 18:07:03.183302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:03.218662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:03.220168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:03.247317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd222e11-2c2f-400d-8e8a-574836f4542c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:03.265923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:03.268172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:03.268588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:03.269925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:03.306036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:03.310313 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:03.332193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 898/1796] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:07:03.441472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 899/1797] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:02 2022] POST /identity/v3/projects => generated 387 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:03.448725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:03.489721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b92ed-697b-4206-ba3d-00e5c03e1e18 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:03.600996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:03.602867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:03.603571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:03.604029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:03.606150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:03.610010 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:03.856961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 899/1798] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:03 2022] POST /identity/v3/users => generated 300 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:03.860512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:03.872896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3c457-8f16-42c8-b8c7-ee72cffd6890 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:03.884676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:03.885832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:03.886120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:03.886394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:03.887880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:03.889924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:04.049426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 900/1799] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:04.058230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:04.069259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a8169-40f5-48d9-ae69-9032ea4c5fe4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:04.078109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:04.079085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:04.079389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:04.079713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] PATH_INFO: `/v3/projects/49c19aa661bd4a3aab227be3c92fae27/users/0eca732e90f34292a2de3be19f629535/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:04.098306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c19aa661bd4a3aab227be3c92fae27, user_id=0eca732e90f34292a2de3be19f629535, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:04.101686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:04.270279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 900/1800] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:04 2022] PUT /identity/v3/projects/49c19aa661bd4a3aab227be3c92fae27/users/0eca732e90f34292a2de3be19f629535/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:04.270584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:04.281893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6f1aa-aee4-4626-82d1-528ee73e8686 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:04.348968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6833c3-03af-4a32-8185-391bf843484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:04.349635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6833c3-03af-4a32-8185-391bf843484a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:04.349913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6833c3-03af-4a32-8185-391bf843484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:04.350186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6833c3-03af-4a32-8185-391bf843484a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:04.350910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6833c3-03af-4a32-8185-391bf843484a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:04.352574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6833c3-03af-4a32-8185-391bf843484a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:04.376397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 901/1801] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:04.378358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:04.389191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70add4c8-348d-4787-9d40-23f81604571f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:04.395716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:04.396773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:04.397054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:04.397336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] PATH_INFO: `/v3/projects/49c19aa661bd4a3aab227be3c92fae27/users/0eca732e90f34292a2de3be19f629535/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:04.399417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c19aa661bd4a3aab227be3c92fae27, user_id=0eca732e90f34292a2de3be19f629535, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:04.402062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:04.415253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 901/1802] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:04 2022] PUT /identity/v3/projects/49c19aa661bd4a3aab227be3c92fae27/users/0eca732e90f34292a2de3be19f629535/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:04.427594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:04.427879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:04.428154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:04.455792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] MFA Rules not processed for user `0eca732e90f34292a2de3be19f629535`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:04.457038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:04.534911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 902/1803] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:04.548458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:04.604823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d587-b7c5-489c-b087-8323af1fd58e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:04.609938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ed7b-0d86-44c4-a193-a27c9e1d0f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:04.611739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ed7b-0d86-44c4-a193-a27c9e1d0f8f service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:04.611739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ed7b-0d86-44c4-a193-a27c9e1d0f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:04.611739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ed7b-0d86-44c4-a193-a27c9e1d0f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:04.623893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4ed7b-0d86-44c4-a193-a27c9e1d0f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:04.625345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4ed7b-0d86-44c4-a193-a27c9e1d0f8f service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:04.637236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 902/1804] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:05.893225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:06.594312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:06.659237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b41c0d-ff4d-4dac-ab8d-7ca33d1b7a38 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:06.663933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:06.664977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:06.665327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:06.665736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] PATH_INFO: `/v3/users/12d7fe122d5742e39422ada4aaabfd10` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:06.667014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d7fe122d5742e39422ada4aaabfd10)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:06.668982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:06.908888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d7fe122d5742e39422ada4aaabfd10'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:06.921267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d7fe122d5742e39422ada4aaabfd10'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:06.928685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d7fe122d5742e39422ada4aaabfd10'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:07.022789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d7fe122d5742e39422ada4aaabfd10'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:07.023465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d7fe122d5742e39422ada4aaabfd10'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:07.030367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 903/1805] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:06 2022] DELETE /identity/v3/users/12d7fe122d5742e39422ada4aaabfd10 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:07.034748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c6be8-de70-4921-af47-33181922958f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:07.035201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c6be8-de70-4921-af47-33181922958f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:07.035305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c6be8-de70-4921-af47-33181922958f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:07.055814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ea3c6be8-de70-4921-af47-33181922958f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:07.057059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ea3c6be8-de70-4921-af47-33181922958f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:07.251738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 903/1806] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:07.259507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:07.260539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:07.343760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e98e1-362b-4cfd-a999-669363ea2e7e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:07.347525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:07.348191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:07.348476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:07.348754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:07.362035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:07.363487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:07.375527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 904/1807] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:07.426338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:07.886380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c6be8-de70-4921-af47-33181922958f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:07.888578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ea3c6be8-de70-4921-af47-33181922958f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:07.921315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c6be8-de70-4921-af47-33181922958f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:07.926050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:07.926934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:07.927219 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:07.927835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] PATH_INFO: `/v3/projects/c528667c6413410b8d3abfb4c8abb4d8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:07.928759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c528667c6413410b8d3abfb4c8abb4d8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:07.930697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:08.055880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c528667c6413410b8d3abfb4c8abb4d8', 'request_id': 'req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.062726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c528667c6413410b8d3abfb4c8abb4d8', 'request_id': 'req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.069539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c528667c6413410b8d3abfb4c8abb4d8', 'request_id': 'req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.075423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:08.179760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] The token cache is being invalidate because project c528667c6413410b8d3abfb4c8abb4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:08.180062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.183244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.183244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 904/1808] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:07 2022] DELETE /identity/v3/projects/c528667c6413410b8d3abfb4c8abb4d8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:08.187148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:08.196985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217d65d-2979-4bfd-bb29-07f3fecbfe25 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:08.258046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:08.263700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:08.263700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:08.263700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] PATH_INFO: `/v3/users/0eca732e90f34292a2de3be19f629535` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:08.263700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eca732e90f34292a2de3be19f629535)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:08.263700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:08.552585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eca732e90f34292a2de3be19f629535'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.566182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eca732e90f34292a2de3be19f629535'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.573349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eca732e90f34292a2de3be19f629535'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.705250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eca732e90f34292a2de3be19f629535'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.705901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78e880e6-0150-4fe9-a63e-f988b0b253f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eca732e90f34292a2de3be19f629535'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:08.712391 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 905/1809] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:08 2022] DELETE /identity/v3/users/0eca732e90f34292a2de3be19f629535 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:09.144826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:09.146279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:09.160550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb3112-8d83-47e4-a64d-3d928a33cd7a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:09.220672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:09.221538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:09.221820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:09.222098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] PATH_INFO: `/v3/projects/49c19aa661bd4a3aab227be3c92fae27` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:09.223259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c19aa661bd4a3aab227be3c92fae27)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:09.225165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:09.356656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c19aa661bd4a3aab227be3c92fae27', 'request_id': 'req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:09.363209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c19aa661bd4a3aab227be3c92fae27', 'request_id': 'req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:09.369581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c19aa661bd4a3aab227be3c92fae27', 'request_id': 'req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:09.510996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] The token cache is being invalidate because project 49c19aa661bd4a3aab227be3c92fae27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:09.511443 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:09.512078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dd60c2b3-35a5-4dd2-a0b3-50efc90259b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:09.513624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 905/1810] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:09 2022] DELETE /identity/v3/projects/49c19aa661bd4a3aab227be3c92fae27 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:09.519154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:09.519630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:09.519757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:09.538964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:09.540214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:09.604932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 906/1811] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:09.613342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:09.613721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:09.614079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:09.634922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:09.636651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:09.647849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 906/1812] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:09.652168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:09.663584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9855380-a447-4ad0-8ed6-1adb6be4aba4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:09.668699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:09.669444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:09.669891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:09.670243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:09.671132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:09.671558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:09.681141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:09.689199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 907/1813] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:09.691843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:09.702713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937bbaf-63d4-4a90-9ce3-1c35bd12ed2b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:09.708879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:09.709223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:09.709610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:09.710061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:09.710915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:09.713228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:09.815238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 907/1814] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:09 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:09.819685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:09.826275 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:09.830614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da010eb-5218-4833-84f0-02c0c987cd59 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:09.888498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:09.889742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:09.890145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:09.890482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:09.896005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:09.896005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:09.992645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 908/1815] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:09 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:09.997200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.008721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ffac-ad9b-4ea1-9ac4-ecc78c60b216 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.013310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.013999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.014514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.014828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.015025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:10.015583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.017257 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.028328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 908/1816] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:10.032378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.042574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe3ab-15a4-4ce6-92ad-2b20deb95503 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.048430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd2b5-3d08-43e2-b680-44d4f5b9f7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.049139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd2b5-3d08-43e2-b680-44d4f5b9f7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.049426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd2b5-3d08-43e2-b680-44d4f5b9f7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.049703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd2b5-3d08-43e2-b680-44d4f5b9f7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.050459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd2b5-3d08-43e2-b680-44d4f5b9f7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.052185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd2b5-3d08-43e2-b680-44d4f5b9f7ce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.064445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 909/1817] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:10.068645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.078949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11039eb-cc7b-4b04-9c1d-8044040e856f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.083928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.084873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.085140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.085408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] PATH_INFO: `/v3/projects/a8c960ddc8434d5197c85794dd155a5d/users/40f50569c83a4515acc5921b686cbf4b/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.100875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c960ddc8434d5197c85794dd155a5d, user_id=40f50569c83a4515acc5921b686cbf4b, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.103593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.179749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 909/1818] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:10 2022] PUT /identity/v3/projects/a8c960ddc8434d5197c85794dd155a5d/users/40f50569c83a4515acc5921b686cbf4b/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:10.191272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.191633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.191916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.219790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] MFA Rules not processed for user `40f50569c83a4515acc5921b686cbf4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:10.221021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:10.287873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 910/1819] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:10.292672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.354470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023bb410-4137-4a34-8db3-a503ca68f54c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.359921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.360580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.360864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.361138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.362080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.364002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.485179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 910/1820] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:10 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:10.489447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.502740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67ace5-11f9-489f-bc5c-a75dffdd1500 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.558272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.558954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.559241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.559552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.560279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.563177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.654721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 911/1821] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:10 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:10.659004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.669156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ee01f-064b-447a-887b-46c4298f3669 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.674976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.675703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.675991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.676275 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.677012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.678648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.699193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 911/1822] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:10.699399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.706722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b6a56-ff12-4a41-aac7-30cc6a972d3a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.711523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.712410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.712707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.712984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] PATH_INFO: `/v3/projects/541ed86b354d47ada8dff46cd361f99b/users/f9bf3d70b8de429795d1d993be2d1f20/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.728591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=541ed86b354d47ada8dff46cd361f99b, user_id=f9bf3d70b8de429795d1d993be2d1f20, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.733017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.761635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:10.820804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 912/1823] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:10 2022] PUT /identity/v3/projects/541ed86b354d47ada8dff46cd361f99b/users/f9bf3d70b8de429795d1d993be2d1f20/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:10.836829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.836829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc1a5d-5b1b-447f-bd78-cf5623cf50aa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.898393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ae110-9d9a-48b5-a0f7-64238613b483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.899347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae110-9d9a-48b5-a0f7-64238613b483 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.902429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae110-9d9a-48b5-a0f7-64238613b483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.902429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ae110-9d9a-48b5-a0f7-64238613b483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.902429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ae110-9d9a-48b5-a0f7-64238613b483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.902683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ae110-9d9a-48b5-a0f7-64238613b483 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.914349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 912/1824] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:10.918867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:10.929550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670d53-9fae-4db9-95b2-53fe09d1b620 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:10.935641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:10.936821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.937243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.937622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] PATH_INFO: `/v3/projects/541ed86b354d47ada8dff46cd361f99b/users/f9bf3d70b8de429795d1d993be2d1f20/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:10.939955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=541ed86b354d47ada8dff46cd361f99b, user_id=f9bf3d70b8de429795d1d993be2d1f20, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:10.942865 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:10.955657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:10.959154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 913/1825] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:10 2022] PUT /identity/v3/projects/541ed86b354d47ada8dff46cd361f99b/users/f9bf3d70b8de429795d1d993be2d1f20/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:10.974622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:10.974917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:10.975201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:11.005053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] MFA Rules not processed for user `f9bf3d70b8de429795d1d993be2d1f20`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:11.006273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:11.083056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 913/1826] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:11.095307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:11.097260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:11.164999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd2e2a-6749-4dbe-a4a0-728e3d95e38e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:11.169097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ada400-6d1a-4db9-803d-230847f196cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:11.169946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada400-6d1a-4db9-803d-230847f196cd service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:11.170373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada400-6d1a-4db9-803d-230847f196cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:11.170674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ada400-6d1a-4db9-803d-230847f196cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:11.228422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ada400-6d1a-4db9-803d-230847f196cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:11.231980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ada400-6d1a-4db9-803d-230847f196cd service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:11.240550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 914/1827] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:11.334199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:11.884175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:12.581373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:12.706975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:12.842161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:13.408456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:13.954104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:15.311615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:19.241389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:19.244347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:19.318650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b22026-5c03-4779-94f4-e3b18a45937a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:19.327348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:19.332254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:19.332254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:19.332254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] PATH_INFO: `/v3/users/560282dbe22b4a3d8195694b7842a7d1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:19.332254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=560282dbe22b4a3d8195694b7842a7d1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:19.336836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:19.481255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6a54-e736-41fe-b051-80091702e2d0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:19.484307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6a54-e736-41fe-b051-80091702e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:19.485763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6a54-e736-41fe-b051-80091702e2d0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:19.492343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 915/1828] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:07:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:07:19.525225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be399-2ba7-4cce-b2ab-05c61f3b2272 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:19.525586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be399-2ba7-4cce-b2ab-05c61f3b2272 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:19.525990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0be399-2ba7-4cce-b2ab-05c61f3b2272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:19.549588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-eb0be399-2ba7-4cce-b2ab-05c61f3b2272 None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:19.551487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eb0be399-2ba7-4cce-b2ab-05c61f3b2272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:19.575611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560282dbe22b4a3d8195694b7842a7d1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:19.577399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560282dbe22b4a3d8195694b7842a7d1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:19.624217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:19.649761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 916/1829] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:07:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:19.691403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560282dbe22b4a3d8195694b7842a7d1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:19.703283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560282dbe22b4a3d8195694b7842a7d1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:19.742882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560282dbe22b4a3d8195694b7842a7d1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:19.751510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 914/1830] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:19 2022] DELETE /identity/v3/users/560282dbe22b4a3d8195694b7842a7d1 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:19.756301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:19.763202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:19.763202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:19.776485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:19.777705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:19.879805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:19.883760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 917/1831] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:19.883889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:19.905303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:19.909602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:19.976021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a411114-237b-4743-9e7e-817f98837f27 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:19.980677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:19.981544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:19.982137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:19.982422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:19.992647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:19.997454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:20.001451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:20.036206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfba6b-4b87-4184-9d76-90bda4d26d7c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:20.056390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:20.057098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:20.057386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:20.057669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:20.058940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 915/1832] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:07:20.063618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:20.078896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:20.081980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ffe23-7e74-4f49-ade1-5d4774029ce6 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:20.094708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 918/1833] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:20.666101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:20.667780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:20.685791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035326-b7ea-4750-b800-49c087ca4478 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:20.692242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:20.694492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:20.694785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:20.695064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] PATH_INFO: `/v3/projects/5e2f354bcbe04a469e80e87fa12e8ca2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:20.696640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e2f354bcbe04a469e80e87fa12e8ca2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:20.698544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:20.918636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e2f354bcbe04a469e80e87fa12e8ca2', 'request_id': 'req-1b076428-2d95-490c-a95b-c51a3f25dd83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:20.925431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e2f354bcbe04a469e80e87fa12e8ca2', 'request_id': 'req-1b076428-2d95-490c-a95b-c51a3f25dd83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:20.932935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e2f354bcbe04a469e80e87fa12e8ca2', 'request_id': 'req-1b076428-2d95-490c-a95b-c51a3f25dd83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:21.031027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] The token cache is being invalidate because project 5e2f354bcbe04a469e80e87fa12e8ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:21.031324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:21.032252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:21.035511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 916/1834] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:20 2022] DELETE /identity/v3/projects/5e2f354bcbe04a469e80e87fa12e8ca2 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:21.041092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955878c-052c-4b83-84bd-30665e0bbfd7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.042021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955878c-052c-4b83-84bd-30665e0bbfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.042460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955878c-052c-4b83-84bd-30665e0bbfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.078587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6955878c-052c-4b83-84bd-30665e0bbfd7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:21.081365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6955878c-052c-4b83-84bd-30665e0bbfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:21.190233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.190233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:21.211740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 919/1835] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:21.220557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.224977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.224977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.246445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:21.247267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:21.265681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 920/1836] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:21.271586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.283012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00cbff-f788-436c-8b11-658f2a17425c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.285142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b076428-2d95-490c-a95b-c51a3f25dd83 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.289727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.290136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.290975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.291358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.291702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.291811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.292496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.292783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.293268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:21.293711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:21.294973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:21.295414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:21.311573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 921/1837] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:21.314046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.324949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaffc0a-a4d3-43d5-9bd1-08dcbcbf01b8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.331892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.339200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.339200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.339200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.339200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:21.339200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:21.374581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:21.376181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:21.387550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 917/1838] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:07:21.499788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.500165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 922/1839] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:21 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:21.516127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef08dc-85e2-44bd-a272-192936202edb service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.630918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.631966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.632392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.633177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.634819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:21.639307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:21.764900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:21.799379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 918/1840] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:21 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:21.804211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.818953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae0bca-7a12-432c-a622-0a9ea4add34e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.824643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.825461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.825891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.826271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.827463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:21.829984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:21.851000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 923/1841] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:21.854711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.868082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218223c-c3b1-4557-9b54-dafde256616f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.884027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3bace-3d0f-4caa-b1f2-1d7a06292f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.884904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3bace-3d0f-4caa-b1f2-1d7a06292f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.885365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3bace-3d0f-4caa-b1f2-1d7a06292f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.885810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3bace-3d0f-4caa-b1f2-1d7a06292f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.886705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3bace-3d0f-4caa-b1f2-1d7a06292f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:21.889121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3bace-3d0f-4caa-b1f2-1d7a06292f0c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:21.921710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 919/1842] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:21.921820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:21.948447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a927b-e308-4264-aeff-f7ec6af37abf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:21.955300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:21.957567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:21.958232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:21.958890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] PATH_INFO: `/v3/projects/8dc4d3956b7148ec9042560310d6532a/users/5fe14dcfd2a4428aa90e5b49e0b9ece6/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:21.997553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4d3956b7148ec9042560310d6532a, user_id=5fe14dcfd2a4428aa90e5b49e0b9ece6, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:22.002054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:22.134466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 924/1843] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:21 2022] PUT /identity/v3/projects/8dc4d3956b7148ec9042560310d6532a/users/5fe14dcfd2a4428aa90e5b49e0b9ece6/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:22.136703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:22.137144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:22.137832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:22.226123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] MFA Rules not processed for user `5fe14dcfd2a4428aa90e5b49e0b9ece6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:22.227784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:22.372978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 920/1844] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:22.381832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:22.389689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:22.485113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03626352-625f-4bcf-9436-ebd178a68260 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:22.490124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396a6f1-5b1d-4b8f-94df-a2be5849fd89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:22.491251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396a6f1-5b1d-4b8f-94df-a2be5849fd89 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:22.491660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396a6f1-5b1d-4b8f-94df-a2be5849fd89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:22.492075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396a6f1-5b1d-4b8f-94df-a2be5849fd89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:22.504648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396a6f1-5b1d-4b8f-94df-a2be5849fd89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:22.506319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396a6f1-5b1d-4b8f-94df-a2be5849fd89 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:22.529951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 925/1845] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:23.108983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:23.682890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:23.684441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:23.854914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db6ab9-0dad-4f24-ae56-b4254a7be11e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:23.869444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:23.869444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:23.869444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:23.869747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] PATH_INFO: `/v3/users/9b1249cd79404d8ba6ba80fb8ac38535` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:23.870905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b1249cd79404d8ba6ba80fb8ac38535)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:23.872850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:24.427327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b1249cd79404d8ba6ba80fb8ac38535'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:24.428117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b1249cd79404d8ba6ba80fb8ac38535'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:24.589002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b1249cd79404d8ba6ba80fb8ac38535'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:24.600240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b1249cd79404d8ba6ba80fb8ac38535'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:24.634367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b1249cd79404d8ba6ba80fb8ac38535'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:24.644610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 921/1846] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:23 2022] DELETE /identity/v3/users/9b1249cd79404d8ba6ba80fb8ac38535 => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:24.646318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:24.647380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:24.647872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:24.668780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:24.674289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:24.767090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 926/1847] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:24.784686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:24.786673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:24.872997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742295-c1d0-42eb-b09b-4451a0e60e51 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:24.880338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:24.886201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:24.886201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:24.886201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:24.893463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:24.895980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:24.921434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 922/1848] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:26.079005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:26.080822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:26.105317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1e633-8034-4565-b6d8-175e8eea22ab None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:26.124145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:26.125416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:26.125695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:26.126188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] PATH_INFO: `/v3/projects/f091a0b898984656b546e3609f567631` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:26.153493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f091a0b898984656b546e3609f567631)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:26.163504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:26.356491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:26.376122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f091a0b898984656b546e3609f567631', 'request_id': 'req-eb4ec6e1-f72a-4943-bb84-fe619818de39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:26.489053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f091a0b898984656b546e3609f567631', 'request_id': 'req-eb4ec6e1-f72a-4943-bb84-fe619818de39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:26.504668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f091a0b898984656b546e3609f567631', 'request_id': 'req-eb4ec6e1-f72a-4943-bb84-fe619818de39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:26.575756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] The token cache is being invalidate because project f091a0b898984656b546e3609f567631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:26.575756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:26.575756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:26.580252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 927/1849] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:26 2022] DELETE /identity/v3/projects/f091a0b898984656b546e3609f567631 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:26.591479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:26.603121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9464af-3049-49fe-815e-c3ac7cdd7aab service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:26.723959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:26.725241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:26.725803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:26.726299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] PATH_INFO: `/v3/users/85549e8ce42d4425b618227ff51654c7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:26.728005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85549e8ce42d4425b618227ff51654c7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:26.730924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:26.874704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:27.030628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85549e8ce42d4425b618227ff51654c7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:27.031416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85549e8ce42d4425b618227ff51654c7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:27.151424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85549e8ce42d4425b618227ff51654c7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:27.169130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85549e8ce42d4425b618227ff51654c7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:27.212395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e039409e-93d4-47a7-8f61-1a187ba3f42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85549e8ce42d4425b618227ff51654c7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:27.229297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 923/1850] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:26 2022] DELETE /identity/v3/users/85549e8ce42d4425b618227ff51654c7 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:28.060965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:28.062480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:28.077101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ec6e1-f72a-4943-bb84-fe619818de39 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:28.164458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:28.165750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:28.166217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:28.166657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] PATH_INFO: `/v3/projects/08f35610c52c423d80093b7dfbb56640` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:28.168337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f35610c52c423d80093b7dfbb56640)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:28.170319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:28.286646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f35610c52c423d80093b7dfbb56640', 'request_id': 'req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:28.356340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f35610c52c423d80093b7dfbb56640', 'request_id': 'req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:28.363847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f35610c52c423d80093b7dfbb56640', 'request_id': 'req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:28.376619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] The token cache is being invalidate because project 08f35610c52c423d80093b7dfbb56640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:28.377291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:28.378004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa879f3a-a0e4-4c0f-bb22-44125fbbfabf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:28.383469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 928/1851] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:28 2022] DELETE /identity/v3/projects/08f35610c52c423d80093b7dfbb56640 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:28.385282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:28.385744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:28.385979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:28.408085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:28.415717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:28.518081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 924/1852] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:28.522574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:28.524214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:28.528051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:28.563203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:28.564516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:28.575273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 929/1853] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:28.579335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:28.589526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d82dd5-f9ea-4305-8e60-cb8400f72f36 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:28.598091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:28.599133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:28.599828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:28.600184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:28.600933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:28.601422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:28.603262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:28.617498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 925/1854] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:28.636089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:28.648724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec3e34-422f-46df-87a1-609c5bf8edd3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:28.656210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:28.656928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:28.657345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:28.657804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:28.658517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:28.660624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:28.791774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 930/1855] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:28 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:28.793211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:28.805365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f7dc5-86d9-4882-8012-1d0c4fd80233 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:28.898945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:28.899941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:28.900296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:28.900903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:28.901897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:28.905280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.033301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 926/1856] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:28 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:29.038277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.048833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741d1ad-9ace-42a0-9f1a-3d48abedff0c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.054890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.055091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.055525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.055806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.056539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.058992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.070549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 931/1857] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:29.074856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.086277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4fb1d-9c61-4d28-8a87-c0e1996107fb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.091240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c136a14-f98b-4b2a-8ab5-7f836072890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.091984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c136a14-f98b-4b2a-8ab5-7f836072890a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.092271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c136a14-f98b-4b2a-8ab5-7f836072890a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.092550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c136a14-f98b-4b2a-8ab5-7f836072890a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.093303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c136a14-f98b-4b2a-8ab5-7f836072890a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.094976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c136a14-f98b-4b2a-8ab5-7f836072890a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.108774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 927/1858] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:29.113200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.124957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2eb2c-75c0-4c7c-b75e-7edffe750b9d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.133678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.133678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.133678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.134118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] PATH_INFO: `/v3/projects/ffae6cedf69b40f2af9cf7177cad7e47/users/4c268f980ae649619ea19b2ece1c2618/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.154157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffae6cedf69b40f2af9cf7177cad7e47, user_id=4c268f980ae649619ea19b2ece1c2618, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.157002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.193405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:29.259782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 932/1859] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:29 2022] PUT /identity/v3/projects/ffae6cedf69b40f2af9cf7177cad7e47/users/4c268f980ae649619ea19b2ece1c2618/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:29.262106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.262444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.262741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.298347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] MFA Rules not processed for user `4c268f980ae649619ea19b2ece1c2618`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:29.300086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:29.375602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 928/1860] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:29.382509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.448285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550d092-2d79-4af0-b684-be674419b0c5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.453004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.453663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.453948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.454228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.454920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.456836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.544418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 933/1861] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:29 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:29.548482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.570745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27ef1e-7b2f-46fe-b1fa-1b42136d3530 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.590324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.592155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:29.613869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:29.638169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.639125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.639448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.639785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.640532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.642553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.673284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f2712-a9ae-46a0-a1af-376c11afb7dc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.678721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.679651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.679941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.680212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] PATH_INFO: `/v3/users/40f50569c83a4515acc5921b686cbf4b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.681420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40f50569c83a4515acc5921b686cbf4b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.683415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.788551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 929/1862] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:29 2022] POST /identity/v3/users => generated 300 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:29.797953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.801018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb078ec-417c-433d-a4c2-d47d3192b3fc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.806145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.806851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.807139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.807450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.808168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.809733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.826805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 930/1863] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:29.827577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.839065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994b14b-3021-46e9-bf7f-ddfee942bbfd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:29.843865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:29.844755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:29.845074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:29.845354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] PATH_INFO: `/v3/projects/f83772619fa04d6fb458b6b582ec007f/users/012eb0ab80ef4a38ac31941f06ab3539/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:29.860422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83772619fa04d6fb458b6b582ec007f, user_id=012eb0ab80ef4a38ac31941f06ab3539, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:29.863054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:29.955479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 931/1864] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:29 2022] PUT /identity/v3/projects/f83772619fa04d6fb458b6b582ec007f/users/012eb0ab80ef4a38ac31941f06ab3539/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:29.960805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:29.964074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40f50569c83a4515acc5921b686cbf4b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:29.979176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40f50569c83a4515acc5921b686cbf4b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:29.987357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40f50569c83a4515acc5921b686cbf4b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:30.027155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309382e-5f62-4f22-a3d3-802d70cd5497 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:30.032155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:30.032849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.033151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.033392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.034120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:30.035781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:30.047203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 932/1865] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:30.051424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:30.062585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38529fc2-a997-4abe-ad43-0f13527a6301 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:30.068423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea022a7-38e4-4c9b-bb1b-7a0f60eed2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:30.070040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea022a7-38e4-4c9b-bb1b-7a0f60eed2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.070040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea022a7-38e4-4c9b-bb1b-7a0f60eed2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.070245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea022a7-38e4-4c9b-bb1b-7a0f60eed2dc admin admin] PATH_INFO: `/v3/projects/f83772619fa04d6fb458b6b582ec007f/users/012eb0ab80ef4a38ac31941f06ab3539/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.072430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea022a7-38e4-4c9b-bb1b-7a0f60eed2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83772619fa04d6fb458b6b582ec007f, user_id=012eb0ab80ef4a38ac31941f06ab3539, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:30.075100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea022a7-38e4-4c9b-bb1b-7a0f60eed2dc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:30.078437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40f50569c83a4515acc5921b686cbf4b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:30.079786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-55a8d9a1-4949-4455-8ed2-d5a2fec3f8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40f50569c83a4515acc5921b686cbf4b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:30.086796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 933/1866] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:30 2022] PUT /identity/v3/projects/f83772619fa04d6fb458b6b582ec007f/users/012eb0ab80ef4a38ac31941f06ab3539/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:30.097993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 934/1867] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:29 2022] DELETE /identity/v3/users/40f50569c83a4515acc5921b686cbf4b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:30.098099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.098099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.098099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.107938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.107938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.107938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.114070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:30.116666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:30.141090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] MFA Rules not processed for user `012eb0ab80ef4a38ac31941f06ab3539`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:30.142740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:30.201860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 934/1868] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:30.213325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:30.214935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:30.240592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 935/1869] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:30.263153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:30.267110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:30.323496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4355828-ac4b-4ef8-8e9c-1243aa7f84b2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:30.328214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:30.328980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.329452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.329812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.341070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:30.342866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:30.355273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 935/1870] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:30.362140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e6fcd-11d4-4610-b09a-bd70bdd02313 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:30.368062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:30.369661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.369947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.370223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.385530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:30.387436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:30.399504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 936/1871] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:30.504804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:30.507235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:30.527930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3a7c9-a67f-429a-bbf9-de4642aba83b service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:30.535578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:30.537222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:30.537575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:30.538136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] PATH_INFO: `/v3/users/4c268f980ae649619ea19b2ece1c2618` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:30.539334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c268f980ae649619ea19b2ece1c2618)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:30.541458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:30.898590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c268f980ae649619ea19b2ece1c2618'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:30.899723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c268f980ae649619ea19b2ece1c2618'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.017760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:31.019603 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:31.034549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb1d2d-6a9c-41d2-9dbc-4712cbd6ab41 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:31.039031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c268f980ae649619ea19b2ece1c2618'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.041843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:31.044631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:31.044631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:31.044631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] PATH_INFO: `/v3/projects/a8c960ddc8434d5197c85794dd155a5d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:31.046172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c960ddc8434d5197c85794dd155a5d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:31.048386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:31.050910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c268f980ae649619ea19b2ece1c2618'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.064643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8e8dc53-7335-4ebc-9258-8775f2394a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c268f980ae649619ea19b2ece1c2618'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.072432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 936/1872] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:30 2022] DELETE /identity/v3/users/4c268f980ae649619ea19b2ece1c2618 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:31.076939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:31.077232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:31.077532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:31.095653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:31.097047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:31.165741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 937/1873] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:31.172424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:31.173901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:31.206347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c960ddc8434d5197c85794dd155a5d', 'request_id': 'req-aab25aab-719c-4de3-a585-57607c5c59d0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.248959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950346bf-e236-4a88-a659-0d772ac5b910 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:31.296189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c960ddc8434d5197c85794dd155a5d', 'request_id': 'req-aab25aab-719c-4de3-a585-57607c5c59d0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.303968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c960ddc8434d5197c85794dd155a5d', 'request_id': 'req-aab25aab-719c-4de3-a585-57607c5c59d0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.315840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:31.316797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:31.317210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:31.317542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:31.319346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] The token cache is being invalidate because project a8c960ddc8434d5197c85794dd155a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:31.319826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.320662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.329382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 937/1874] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:31 2022] DELETE /identity/v3/projects/a8c960ddc8434d5197c85794dd155a5d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:31.336994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:31.338424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:31.360428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:31.397749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:31.399233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:31.423534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 938/1875] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:07:31.429560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab25aab-719c-4de3-a585-57607c5c59d0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:31.435680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:31.436583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:31.436871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:31.437157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] PATH_INFO: `/v3/users/f9bf3d70b8de429795d1d993be2d1f20` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:31.440317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9bf3d70b8de429795d1d993be2d1f20)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:31.442619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:31.759788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9bf3d70b8de429795d1d993be2d1f20'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.776585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9bf3d70b8de429795d1d993be2d1f20'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.785789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9bf3d70b8de429795d1d993be2d1f20'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.908788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9bf3d70b8de429795d1d993be2d1f20'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.909776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9bf3d70b8de429795d1d993be2d1f20'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:31.922251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 938/1876] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:31 2022] DELETE /identity/v3/users/f9bf3d70b8de429795d1d993be2d1f20 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:32.022754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:32.184545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:32.185998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:32.200045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbbf87-fa4c-4ac4-a889-b1f0464dd501 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:32.296560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:32.298744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:32.298744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:32.298918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] PATH_INFO: `/v3/projects/ffae6cedf69b40f2af9cf7177cad7e47` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:32.300180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffae6cedf69b40f2af9cf7177cad7e47)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:32.302434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:32.463759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffae6cedf69b40f2af9cf7177cad7e47', 'request_id': 'req-959e7af0-e5de-4183-87f2-6dd1ceb58318'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.469664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffae6cedf69b40f2af9cf7177cad7e47', 'request_id': 'req-959e7af0-e5de-4183-87f2-6dd1ceb58318'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.478193 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffae6cedf69b40f2af9cf7177cad7e47', 'request_id': 'req-959e7af0-e5de-4183-87f2-6dd1ceb58318'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.533736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:32.538315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:32.558290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1fb07-abfa-4769-87e5-2c136e7991dc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:32.590934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] The token cache is being invalidate because project ffae6cedf69b40f2af9cf7177cad7e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:32.591292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.592142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.594406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 939/1877] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:32 2022] DELETE /identity/v3/projects/ffae6cedf69b40f2af9cf7177cad7e47 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:32.601531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:32.603254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:32.637885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:32.638770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:32.639059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:32.639345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] PATH_INFO: `/v3/projects/541ed86b354d47ada8dff46cd361f99b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:32.640605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=541ed86b354d47ada8dff46cd361f99b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:32.643039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:32.698920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e7af0-e5de-4183-87f2-6dd1ceb58318 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:32.703714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:32.704607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:32.704900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:32.705178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] PATH_INFO: `/v3/users/012eb0ab80ef4a38ac31941f06ab3539` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:32.706593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=012eb0ab80ef4a38ac31941f06ab3539)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:32.708633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:32.838182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '541ed86b354d47ada8dff46cd361f99b', 'request_id': 'req-85d79637-14b9-4822-ae57-2413897eaaaf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.976854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '541ed86b354d47ada8dff46cd361f99b', 'request_id': 'req-85d79637-14b9-4822-ae57-2413897eaaaf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.987042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '541ed86b354d47ada8dff46cd361f99b', 'request_id': 'req-85d79637-14b9-4822-ae57-2413897eaaaf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:32.999420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] The token cache is being invalidate because project 541ed86b354d47ada8dff46cd361f99b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:32.999978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.000868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85d79637-14b9-4822-ae57-2413897eaaaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.003221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 939/1878] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:32 2022] DELETE /identity/v3/projects/541ed86b354d47ada8dff46cd361f99b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:33.012279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ba495-f5f9-4120-9a10-6a00c31e3916 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.013074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ba495-f5f9-4120-9a10-6a00c31e3916 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.013462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ba495-f5f9-4120-9a10-6a00c31e3916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.034561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012eb0ab80ef4a38ac31941f06ab3539'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.035335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012eb0ab80ef4a38ac31941f06ab3539'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.036989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6b2ba495-f5f9-4120-9a10-6a00c31e3916 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:33.038501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6b2ba495-f5f9-4120-9a10-6a00c31e3916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:33.117718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 940/1879] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:33.126551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.126852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.127165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.154857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:33.156228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:33.159811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012eb0ab80ef4a38ac31941f06ab3539'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.169232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 941/1880] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:07:33.169891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012eb0ab80ef4a38ac31941f06ab3539'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.173612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.184798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddda77a-f1d1-4b0b-b540-1c9ac21d21dd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.189594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:33.190538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.190833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.191150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.192095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:33.192418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:33.194388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:33.199578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012eb0ab80ef4a38ac31941f06ab3539'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:33.207191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 942/1881] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:33.212376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 940/1882] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:32 2022] DELETE /identity/v3/users/012eb0ab80ef4a38ac31941f06ab3539 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:33.213156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.214919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:33.229672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a96472-bc2c-4c70-b267-269e45572101 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.305232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:33.305929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.306218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.306495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.307630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:33.309828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:33.478959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 943/1883] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:33 2022] POST /identity/v3/projects => generated 371 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:33.483691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.494362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c54560-5b23-432c-a3a1-172f266c1679 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.561652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:33.562360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.562647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.562974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.563682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:33.565731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:33.661463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 941/1884] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:33 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:33.663706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.676721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eaebd-c0d0-41ef-8264-386d8b8ee6ce admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.685608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:33.686364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.686657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.686943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.687792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:33.689451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:33.703753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 944/1885] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:33.705058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.715440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6327f-9d3e-4fb6-9bcc-b1c81185bbb3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.721189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:33.722141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.722482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.722766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.723603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:33.728198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:33.739500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 942/1886] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:33.742809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.766486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc5ba2-9e73-4c78-ab44-9ca3f4fa7635 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.780234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024187b-7eff-4856-9666-0698774e2239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:33.781202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024187b-7eff-4856-9666-0698774e2239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.781488 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024187b-7eff-4856-9666-0698774e2239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.781762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024187b-7eff-4856-9666-0698774e2239 admin admin] PATH_INFO: `/v3/projects/7385d827292c44cf91ca0689dfc3753a/users/84114b0e2d6d4ea4b0b494d8b6e66a0e/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.797174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:33.815538 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024187b-7eff-4856-9666-0698774e2239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7385d827292c44cf91ca0689dfc3753a, user_id=84114b0e2d6d4ea4b0b494d8b6e66a0e, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:33.818376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024187b-7eff-4856-9666-0698774e2239 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:33.924231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:33.926169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 945/1887] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:33 2022] PUT /identity/v3/projects/7385d827292c44cf91ca0689dfc3753a/users/84114b0e2d6d4ea4b0b494d8b6e66a0e/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:33.927232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:33.938500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:33.938500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:33.938500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:33.947844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:33.950838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2554fa-b29d-4938-9ca5-b4e9f5b56bcf admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:33.958689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:34.046585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 946/1888] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:34.046780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:34.047826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.048114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.048386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] PATH_INFO: `/v3/projects/f83772619fa04d6fb458b6b582ec007f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.050121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f83772619fa04d6fb458b6b582ec007f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:34.052092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:34.056737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:34.058126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:34.155255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c747e7-1724-4f66-818a-e74d796f64d2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:34.159753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c280a03-1298-4fa5-b264-99059f0d0b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:34.160854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280a03-1298-4fa5-b264-99059f0d0b0d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.161212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280a03-1298-4fa5-b264-99059f0d0b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.161560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280a03-1298-4fa5-b264-99059f0d0b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.185411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280a03-1298-4fa5-b264-99059f0d0b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:34.187465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280a03-1298-4fa5-b264-99059f0d0b0d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:34.201011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 947/1889] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:34.275156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f83772619fa04d6fb458b6b582ec007f', 'request_id': 'req-34702018-45ec-4c8a-b25f-f519e4b7910b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:34.282615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f83772619fa04d6fb458b6b582ec007f', 'request_id': 'req-34702018-45ec-4c8a-b25f-f519e4b7910b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:34.296923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f83772619fa04d6fb458b6b582ec007f', 'request_id': 'req-34702018-45ec-4c8a-b25f-f519e4b7910b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:34.428695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] The token cache is being invalidate because project f83772619fa04d6fb458b6b582ec007f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:34.428981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:34.433328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34702018-45ec-4c8a-b25f-f519e4b7910b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:34.435428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 943/1890] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:33 2022] DELETE /identity/v3/projects/f83772619fa04d6fb458b6b582ec007f => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:34.441123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.445171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.446001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.469541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:34.483889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:34.484109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:34.597710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.598100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.598492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.603823 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 948/1891] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:34.628842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:34.631415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:34.641891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 944/1892] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:34.645884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:34.657154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5d18a-9bbd-4677-866b-98f44414f72c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:34.662253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:34.663897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.663897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.664595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.665028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:34.665334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:34.666945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:34.682510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 949/1893] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:34.684207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:34.693499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641dd456-d6af-4952-ad73-a15a8b264dff None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:34.700115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:34.700795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.701226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.701427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.702130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:34.704103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:34.838204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 945/1894] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:34 2022] POST /identity/v3/projects => generated 393 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:34.842504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:34.854000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aaf493-058d-42b1-8ee8-5188230542a0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:34.923843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:34.924524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:34.924812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:34.925086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:34.925787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:34.927743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.061176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 950/1895] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:34 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:35.065037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.076186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b728eb-b7f7-4d58-9860-5397c523af5f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.081059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.081739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.082026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.082303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.083028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.085394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.096947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 946/1896] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:35.102168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.112716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a43f4-22a1-4602-a939-66198c44e00f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.118058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a615ff1-eb14-4641-807d-689c4c0de80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.118731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a615ff1-eb14-4641-807d-689c4c0de80d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.119022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a615ff1-eb14-4641-807d-689c4c0de80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.119302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a615ff1-eb14-4641-807d-689c4c0de80d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.120069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a615ff1-eb14-4641-807d-689c4c0de80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.122062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a615ff1-eb14-4641-807d-689c4c0de80d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.134245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 951/1897] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:35.138561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.150344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ca066-388d-4fc2-bd62-1bfabf20f0de admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.155154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.156081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.156365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.156642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] PATH_INFO: `/v3/projects/83b787a32592494089185b0465e42dbf/users/73d3efd089894a01b4a4ff8c67832738/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.174008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b787a32592494089185b0465e42dbf, user_id=73d3efd089894a01b4a4ff8c67832738, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.176830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.307187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 947/1898] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:35 2022] PUT /identity/v3/projects/83b787a32592494089185b0465e42dbf/users/73d3efd089894a01b4a4ff8c67832738/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:35.317986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.318279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.318568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.375843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] MFA Rules not processed for user `73d3efd089894a01b4a4ff8c67832738`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:35.377077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:35.450034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 952/1899] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:35.454522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.530138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99feb-4f9f-4fcb-a2a4-8a3593db3325 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.535083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.535818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.536108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.536385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.537084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.539599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.691997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 948/1900] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:35 2022] POST /identity/v3/projects => generated 389 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:35.697130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.710629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9969d0-2e25-4caf-9608-ba1025e5168f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.773349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.774133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.774421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.774701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.775421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.777475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.800869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:35.905166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 953/1901] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:35 2022] POST /identity/v3/users => generated 301 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:35.909467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.920845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4178d0-7280-4f4d-90c1-10c74b8795c1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.926427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.927453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.927885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.928311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.930392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.932920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:35.947780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 949/1902] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:35.951981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:35.966759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2eb8aa-6eb1-49ea-8d59-309e560135c3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:35.972280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68152e9-a5e9-4448-b948-a0f26ed19bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:35.978725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68152e9-a5e9-4448-b948-a0f26ed19bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:35.978725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68152e9-a5e9-4448-b948-a0f26ed19bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:35.978725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68152e9-a5e9-4448-b948-a0f26ed19bad admin admin] PATH_INFO: `/v3/projects/f58a8494cd4d4352bd0ecd7257432b5a/users/816c8a86b1ec41d7b223d71bee56dacc/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:35.990579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68152e9-a5e9-4448-b948-a0f26ed19bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58a8494cd4d4352bd0ecd7257432b5a, user_id=816c8a86b1ec41d7b223d71bee56dacc, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:35.993455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68152e9-a5e9-4448-b948-a0f26ed19bad admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:36.041571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:36.113373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 954/1903] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:35 2022] PUT /identity/v3/projects/f58a8494cd4d4352bd0ecd7257432b5a/users/816c8a86b1ec41d7b223d71bee56dacc/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:36.118589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:36.137556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e27b5-de75-44be-8e70-d14f89d6e2d2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:36.159452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.159988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.160458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.197306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] MFA Rules not processed for user `84114b0e2d6d4ea4b0b494d8b6e66a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:36.198738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:36.270383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:36.271078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.271414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.271715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.274434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:36.279637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:36.297029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 950/1904] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:36.298091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:36.319837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7a2c8-8de3-43af-8b46-f038c68d51ba admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:36.335251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b69228-1626-40b3-93d0-f9208ebd3c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:36.336544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69228-1626-40b3-93d0-f9208ebd3c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.336837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69228-1626-40b3-93d0-f9208ebd3c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.337121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69228-1626-40b3-93d0-f9208ebd3c74 admin admin] PATH_INFO: `/v3/projects/f58a8494cd4d4352bd0ecd7257432b5a/users/816c8a86b1ec41d7b223d71bee56dacc/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.345361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69228-1626-40b3-93d0-f9208ebd3c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58a8494cd4d4352bd0ecd7257432b5a, user_id=816c8a86b1ec41d7b223d71bee56dacc, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:36.348338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69228-1626-40b3-93d0-f9208ebd3c74 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:36.356136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 955/1905] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:36.385378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 951/1906] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:36 2022] PUT /identity/v3/projects/f58a8494cd4d4352bd0ecd7257432b5a/users/816c8a86b1ec41d7b223d71bee56dacc/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:36.385621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:36.385621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:36.389660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.390213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.392870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.477955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbc0b-9a97-48c9-980a-c4d451398195 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:36.512190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] MFA Rules not processed for user `816c8a86b1ec41d7b223d71bee56dacc`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:36.515414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:36.565202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:36.565794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.566773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.567066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.613854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 952/1907] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:36.625982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:36.628269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:36.636809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:36.641759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:36.641892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaebeac-0279-4bb6-ace8-9fac7f8d0c38 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:36.645468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63fecb-0b0d-436a-8d56-b3aa6b8e3ff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:36.646216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63fecb-0b0d-436a-8d56-b3aa6b8e3ff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.646562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63fecb-0b0d-436a-8d56-b3aa6b8e3ff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.646903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63fecb-0b0d-436a-8d56-b3aa6b8e3ff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.650239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 956/1908] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:36.659239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63fecb-0b0d-436a-8d56-b3aa6b8e3ff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:36.660753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63fecb-0b0d-436a-8d56-b3aa6b8e3ff0 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:36.672478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 953/1909] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:07:36.719453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:36.729251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55304c-ba65-4b6f-9e3c-0e03d52394e8 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:36.733332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:36.737098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:36.737098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:36.737098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:36.798948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:36.800494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:36.811098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 957/1910] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:38.132856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:38.190417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:38.514198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:39.216978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:39.447223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:39.595794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:39.692994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:39.727604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:42.784860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e542d6-1575-4426-85d5-989ef5480895 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:42.785235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e542d6-1575-4426-85d5-989ef5480895 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:42.785235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e542d6-1575-4426-85d5-989ef5480895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:42.804109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d9e542d6-1575-4426-85d5-989ef5480895 None None] MFA Rules not processed for user `84114b0e2d6d4ea4b0b494d8b6e66a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:42.805424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d9e542d6-1575-4426-85d5-989ef5480895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:42.815030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 954/1911] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:49.278071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:49.345819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104573ce-8573-4594-bf33-b42a1f8c37bf service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:49.350767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:49.354959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:49.354959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:49.354959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] PATH_INFO: `/v3/users/73d3efd089894a01b4a4ff8c67832738` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:49.354959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d3efd089894a01b4a4ff8c67832738)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:49.356732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:49.601141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d3efd089894a01b4a4ff8c67832738'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:49.615062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d3efd089894a01b4a4ff8c67832738'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:49.623251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d3efd089894a01b4a4ff8c67832738'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:49.678786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d3efd089894a01b4a4ff8c67832738'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:49.679523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d3efd089894a01b4a4ff8c67832738'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:49.687335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 958/1912] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:49 2022] DELETE /identity/v3/users/73d3efd089894a01b4a4ff8c67832738 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:49.692243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:49.692748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:49.692847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:49.712008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:49.713576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:49.780962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 955/1913] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:49.819790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:49.821765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:49.835970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:49.900444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73517f05-c96c-4f69-92aa-fe1b3ee61e35 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:49.904831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:49.905496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:49.905782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:49.906062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:49.917312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:49.919856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:49.931465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 959/1914] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:50.686331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:50.687954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:50.703221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f9d2d-dc65-4af1-9ee0-4043b580afe7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:50.710978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:50.714107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:50.714415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:50.714743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] PATH_INFO: `/v3/projects/83b787a32592494089185b0465e42dbf` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:50.716201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b787a32592494089185b0465e42dbf)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:50.718851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:50.842998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:50.846442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b787a32592494089185b0465e42dbf', 'request_id': 'req-f93e82c8-2921-40fd-871b-a77eb4d51855'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:50.854017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b787a32592494089185b0465e42dbf', 'request_id': 'req-f93e82c8-2921-40fd-871b-a77eb4d51855'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:50.864469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b787a32592494089185b0465e42dbf', 'request_id': 'req-f93e82c8-2921-40fd-871b-a77eb4d51855'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:50.955641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] The token cache is being invalidate because project 83b787a32592494089185b0465e42dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:50.958309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:50.958309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:50.958861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 956/1915] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:50 2022] DELETE /identity/v3/projects/83b787a32592494089185b0465e42dbf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:50.962932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:50.977228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05535422-9caa-4b15-ba1b-a220fc4b9460 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:51.054803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:51.055811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:51.056330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:51.056615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] PATH_INFO: `/v3/users/816c8a86b1ec41d7b223d71bee56dacc` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:51.057836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] RBAC: Authorizing `identity:delete_user(user_id=816c8a86b1ec41d7b223d71bee56dacc)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:51.060227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:51.339755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816c8a86b1ec41d7b223d71bee56dacc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:51.355021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816c8a86b1ec41d7b223d71bee56dacc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:51.378609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816c8a86b1ec41d7b223d71bee56dacc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:51.491424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816c8a86b1ec41d7b223d71bee56dacc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:51.492145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ebb0eded-f452-41fd-8bca-25636169678c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816c8a86b1ec41d7b223d71bee56dacc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:51.500465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 960/1916] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:50 2022] DELETE /identity/v3/users/816c8a86b1ec41d7b223d71bee56dacc => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:52.011045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:52.012575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:52.029691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e82c8-2921-40fd-871b-a77eb4d51855 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:52.092841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:52.093715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:52.094001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:52.094279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] PATH_INFO: `/v3/projects/f58a8494cd4d4352bd0ecd7257432b5a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:52.095530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58a8494cd4d4352bd0ecd7257432b5a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:52.098340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:52.229788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58a8494cd4d4352bd0ecd7257432b5a', 'request_id': 'req-f0b8ed25-29b3-4385-8ca0-1352884997b0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:52.235525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58a8494cd4d4352bd0ecd7257432b5a', 'request_id': 'req-f0b8ed25-29b3-4385-8ca0-1352884997b0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:52.245973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58a8494cd4d4352bd0ecd7257432b5a', 'request_id': 'req-f0b8ed25-29b3-4385-8ca0-1352884997b0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:52.425926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] The token cache is being invalidate because project f58a8494cd4d4352bd0ecd7257432b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:52.425926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:52.425926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f0b8ed25-29b3-4385-8ca0-1352884997b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:52.425926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 957/1917] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:52 2022] DELETE /identity/v3/projects/f58a8494cd4d4352bd0ecd7257432b5a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:52.426562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:52.426562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:52.426562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:52.435893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:52.495261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:52.495261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:52.509593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 961/1918] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:52.518601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:52.519135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:52.519671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:52.569592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:52.572036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:52.584325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 958/1919] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:52.588778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:52.602076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bff2d4-6e89-43a3-9e69-b44819e7c3ed None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:52.608356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:52.609207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:52.609781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:52.610225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:52.611181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:52.611714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:52.613410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:52.629742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 962/1920] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:52.633912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:52.644562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655b269-6f58-4779-9efc-2fd2311a35ca None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:52.651244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:52.652344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:52.652867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:52.653245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:52.654027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:52.656124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:53.010399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 959/1921] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:52 2022] POST /identity/v3/projects => generated 383 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:53.017029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:53.030727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cbce1-e981-4751-820c-c1051373b0d3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:53.105060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:53.108715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.108715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.108715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.108715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:53.109409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:53.199082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 963/1922] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:53 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:53.204554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:53.215033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646c2dc-d845-48aa-bba6-7f62cc66606a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:53.220921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:53.221604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.221893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.222355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.223254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:53.224980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:53.237728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 960/1923] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:07:53.243978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:53.257023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164ccd0-579b-4e63-b39f-498aec8bd45a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:53.262188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a163f4-49c2-4b3a-98e6-179b450098e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:53.265092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a163f4-49c2-4b3a-98e6-179b450098e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.265386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a163f4-49c2-4b3a-98e6-179b450098e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.265666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a163f4-49c2-4b3a-98e6-179b450098e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.267103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a163f4-49c2-4b3a-98e6-179b450098e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:53.270945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a163f4-49c2-4b3a-98e6-179b450098e8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:53.285804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 964/1924] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:53.290416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:53.302755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74c206-d305-498b-a6bb-629731c419d4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:53.307416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:53.308349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.308732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.308992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] PATH_INFO: `/v3/projects/9c1041d3ae8e4128a869ef85fe73d5a2/users/fce5533d4d94463786fc5c4b31adb466/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.333353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1041d3ae8e4128a869ef85fe73d5a2, user_id=fce5533d4d94463786fc5c4b31adb466, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:53.336276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:53.469737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:53.475724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 961/1925] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:53 2022] PUT /identity/v3/projects/9c1041d3ae8e4128a869ef85fe73d5a2/users/fce5533d4d94463786fc5c4b31adb466/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:53.496310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.496310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.496310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.551492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] MFA Rules not processed for user `fce5533d4d94463786fc5c4b31adb466`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:53.551492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:53.660210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 965/1926] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:53.661717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:53.726584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f53def-40f0-41a7-8575-9350b8cde651 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:53.731211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:53.732568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.740045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.740045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.740045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:53.740045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:53.830218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 962/1927] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:53 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:07:53.834731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:53.847199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847803b1-731b-47f2-86b1-c09d0fadbbfd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:53.878413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:53.914892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:53.914892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:53.914892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:53.914892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:53.915223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:53.915223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:54.048411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 966/1928] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:53 2022] POST /identity/v3/users => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:54.054805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:54.065611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69c0bb-2f4a-4833-b160-a9c2d1ffb593 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:54.070390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:54.071123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:54.071459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:54.071809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:54.072544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:54.074266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:54.086345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 963/1929] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:54.090766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:54.102157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd19d4-09a9-4ee4-b114-cd73d69e7d03 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:54.109749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:54.110839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:54.111123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:54.111428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] PATH_INFO: `/v3/projects/faa09ba1777d43b2a6cee7c71a5c59e1/users/154a863d96e94a05a3e5438f8f4756dc/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:54.138856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa09ba1777d43b2a6cee7c71a5c59e1, user_id=154a863d96e94a05a3e5438f8f4756dc, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:54.138856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:54.293275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 967/1930] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:54 2022] PUT /identity/v3/projects/faa09ba1777d43b2a6cee7c71a5c59e1/users/154a863d96e94a05a3e5438f8f4756dc/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:54.299763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:54.308393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edf9a1-f6d9-4396-8acd-8aa9cd2ce7dd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:54.375385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e4ac3-cbe4-4a97-86c8-c67a265fe743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:54.376224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e4ac3-cbe4-4a97-86c8-c67a265fe743 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:54.376517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e4ac3-cbe4-4a97-86c8-c67a265fe743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:54.376796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e4ac3-cbe4-4a97-86c8-c67a265fe743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:54.378168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e4ac3-cbe4-4a97-86c8-c67a265fe743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:54.380640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e4ac3-cbe4-4a97-86c8-c67a265fe743 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:54.406960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 964/1931] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:54.411482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:54.423294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c17be-e07c-4630-96cc-5211c60058ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:54.428281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:54.429182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:54.429879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:54.429879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] PATH_INFO: `/v3/projects/faa09ba1777d43b2a6cee7c71a5c59e1/users/154a863d96e94a05a3e5438f8f4756dc/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:54.431858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa09ba1777d43b2a6cee7c71a5c59e1, user_id=154a863d96e94a05a3e5438f8f4756dc, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:54.434677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:54.462057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:54.462681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:54.463245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:54.464756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 968/1932] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:54 2022] PUT /identity/v3/projects/faa09ba1777d43b2a6cee7c71a5c59e1/users/154a863d96e94a05a3e5438f8f4756dc/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:54.506605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] MFA Rules not processed for user `154a863d96e94a05a3e5438f8f4756dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:54.508496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:54.614801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 965/1933] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:54.670706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:54.674822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:54.752296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c01544-0607-4fa4-9441-f53d833b2a0c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:54.758315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:54.759242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:54.759651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:54.760032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:54.774630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:54.776449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:54.795735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 969/1934] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:55.007150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:55.009094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:55.039339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57eb4e-5c47-4c4c-963f-26d14c2cb422 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:55.043917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19ac65c-9b41-4167-b6a9-4dbe6351f054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:55.046682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ac65c-9b41-4167-b6a9-4dbe6351f054 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:55.048744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ac65c-9b41-4167-b6a9-4dbe6351f054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:55.049295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19ac65c-9b41-4167-b6a9-4dbe6351f054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:55.164693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ac65c-9b41-4167-b6a9-4dbe6351f054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:55.166178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19ac65c-9b41-4167-b6a9-4dbe6351f054 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:55.179024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 966/1935] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:55.189214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:55.747476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:55.829786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871af51-16c2-43cc-80a2-88af9dd83cb4 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:55.834444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:55.835250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:55.835579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:55.835855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] PATH_INFO: `/v3/users/fce5533d4d94463786fc5c4b31adb466` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:55.837045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce5533d4d94463786fc5c4b31adb466)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:55.839008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:55.864170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:56.017108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce5533d4d94463786fc5c4b31adb466'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:56.035470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce5533d4d94463786fc5c4b31adb466'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:56.042411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce5533d4d94463786fc5c4b31adb466'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:56.147388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce5533d4d94463786fc5c4b31adb466'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:56.147967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce5533d4d94463786fc5c4b31adb466'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:56.155017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 970/1936] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:55 2022] DELETE /identity/v3/users/fce5533d4d94463786fc5c4b31adb466 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:56.159792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:56.160333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:56.160469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:56.178230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:56.179689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:56.259994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 967/1937] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:56.271299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:56.276567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:56.339116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0ce1b-1383-4e17-af4a-dc63ea499e0b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:56.343700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:56.344464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:56.344820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:56.345170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:56.356015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:56.358354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:56.373407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 971/1938] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:07:56.930597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:56.931844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:56.948204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5b31-7593-40d8-b4eb-1eeda658e4d1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:56.956076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:56.957292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:56.957764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:56.958372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] PATH_INFO: `/v3/projects/9c1041d3ae8e4128a869ef85fe73d5a2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:56.960624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1041d3ae8e4128a869ef85fe73d5a2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:56.962885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:57.139176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1041d3ae8e4128a869ef85fe73d5a2', 'request_id': 'req-4d7d73db-f689-490e-a426-d8b81427c14e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.144894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1041d3ae8e4128a869ef85fe73d5a2', 'request_id': 'req-4d7d73db-f689-490e-a426-d8b81427c14e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.151098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1041d3ae8e4128a869ef85fe73d5a2', 'request_id': 'req-4d7d73db-f689-490e-a426-d8b81427c14e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.240591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] The token cache is being invalidate because project 9c1041d3ae8e4128a869ef85fe73d5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:57.241236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.241599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.243245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 968/1939] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:56 2022] DELETE /identity/v3/projects/9c1041d3ae8e4128a869ef85fe73d5a2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:57.247431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:57.263794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79ab59-51aa-472b-8b78-2872c28017c4 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:57.325569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:57.326454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:57.326736 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:57.327015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] PATH_INFO: `/v3/users/154a863d96e94a05a3e5438f8f4756dc` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:57.328245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=154a863d96e94a05a3e5438f8f4756dc)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:57.330243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:57.537621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:57.539396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154a863d96e94a05a3e5438f8f4756dc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.552678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154a863d96e94a05a3e5438f8f4756dc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.560109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154a863d96e94a05a3e5438f8f4756dc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.631241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154a863d96e94a05a3e5438f8f4756dc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.632844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-365e15cc-bb7c-4478-b521-a66756e64e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154a863d96e94a05a3e5438f8f4756dc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:57.639252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 972/1940] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:07:57 2022] DELETE /identity/v3/users/154a863d96e94a05a3e5438f8f4756dc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:07:58.140375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:58.142054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:58.156486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d73db-f689-490e-a426-d8b81427c14e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:58.174458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:58.228945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:58.229269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:58.229515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:58.229804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] PATH_INFO: `/v3/projects/faa09ba1777d43b2a6cee7c71a5c59e1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:58.231038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa09ba1777d43b2a6cee7c71a5c59e1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:58.233114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:58.419639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa09ba1777d43b2a6cee7c71a5c59e1', 'request_id': 'req-0b614049-50c8-4920-8c46-b13b3a6b31d3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:58.430258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa09ba1777d43b2a6cee7c71a5c59e1', 'request_id': 'req-0b614049-50c8-4920-8c46-b13b3a6b31d3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:58.439704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa09ba1777d43b2a6cee7c71a5c59e1', 'request_id': 'req-0b614049-50c8-4920-8c46-b13b3a6b31d3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:58.553444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] The token cache is being invalidate because project faa09ba1777d43b2a6cee7c71a5c59e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:07:58.554226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:58.554342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b614049-50c8-4920-8c46-b13b3a6b31d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:07:58.555984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 969/1941] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:07:58 2022] DELETE /identity/v3/projects/faa09ba1777d43b2a6cee7c71a5c59e1 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:58.561702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:58.562218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:58.562320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:58.581635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:58.582884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:58.653971 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 973/1942] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:58.663486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:58.663486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:58.663486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:58.682054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:58.683409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:58.693692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 970/1943] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:58.698033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:58.709578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189bbd-b2f6-4a45-8b89-4194de52aff3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:58.714814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:58.715589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:58.715946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:58.716253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:58.716974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:07:58.717591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:58.719335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:58.731418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 974/1944] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:07:58.735635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:58.745279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586a9e4-33b6-4342-9523-74e4bde21d79 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:58.751340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:58.752096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:58.752412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:58.752691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:58.753697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:58.755776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:58.860859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 971/1945] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:07:58 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:58.861026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:58.869312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee1119-6ba2-4782-9194-0f52ce4f7098 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:58.939081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:58.939779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:58.940065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:58.940339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:58.941036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:58.942931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:59.042293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 975/1946] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:07:58 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:07:59.047499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:59.058451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c110a-35ca-4091-a3b0-fa772ac1bdeb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:59.064337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:59.065035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:59.065321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:59.065601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:59.066330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:59.068066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:59.079644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 972/1947] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:59.083694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:59.094126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd6a74-f6bf-434c-a9cf-d4fa2c8c7f5c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:59.099144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95212442-934f-433e-9571-0116151af250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:59.099842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95212442-934f-433e-9571-0116151af250 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:59.100135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95212442-934f-433e-9571-0116151af250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:59.100420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95212442-934f-433e-9571-0116151af250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:59.101172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95212442-934f-433e-9571-0116151af250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:59.102841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95212442-934f-433e-9571-0116151af250 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:59.113931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 976/1948] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:07:59.136849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:59.146688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ec19f-fe0e-457a-b72e-0f09e02e9784 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:59.151585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:59.152475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:59.152757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:59.153038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] PATH_INFO: `/v3/projects/bccf2c43b8444d94bd8727cf3ad2730b/users/306d12450e494912a20cf6b5c19efa50/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:59.167953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccf2c43b8444d94bd8727cf3ad2730b, user_id=306d12450e494912a20cf6b5c19efa50, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:59.173780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:59.308449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 973/1949] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:07:59 2022] PUT /identity/v3/projects/bccf2c43b8444d94bd8727cf3ad2730b/users/306d12450e494912a20cf6b5c19efa50/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:07:59.320447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:59.320810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:59.321043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:59.357489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] MFA Rules not processed for user `306d12450e494912a20cf6b5c19efa50`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:07:59.367557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:59.384115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:59.439424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 977/1950] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:07:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:07:59.450149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:07:59.451922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:07:59.513803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:07:59.524607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8b3c4-52af-4632-9b5f-d0c4dd7366b9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:07:59.528317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08135c0b-d96e-49f0-b53a-b40e4decbb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:07:59.528973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08135c0b-d96e-49f0-b53a-b40e4decbb1d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:07:59.529263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08135c0b-d96e-49f0-b53a-b40e4decbb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:07:59.529546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08135c0b-d96e-49f0-b53a-b40e4decbb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:07:59.539258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08135c0b-d96e-49f0-b53a-b40e4decbb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:07:59.540733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08135c0b-d96e-49f0-b53a-b40e4decbb1d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:07:59.550894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 974/1951] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:07:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:04.795146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:04.862962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261c1f6-90c3-4346-864c-ebc26dfd8279 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:04.867454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:04.868321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:04.868604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:04.868885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] PATH_INFO: `/v3/users/306d12450e494912a20cf6b5c19efa50` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:04.870163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306d12450e494912a20cf6b5c19efa50)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:04.873643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:05.077540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306d12450e494912a20cf6b5c19efa50'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:05.090706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306d12450e494912a20cf6b5c19efa50'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:05.098722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306d12450e494912a20cf6b5c19efa50'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:05.161268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306d12450e494912a20cf6b5c19efa50'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:05.161974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306d12450e494912a20cf6b5c19efa50'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:05.168995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 978/1952] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:04 2022] DELETE /identity/v3/users/306d12450e494912a20cf6b5c19efa50 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:05.174525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:05.174990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:05.175561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:05.194963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:05.196843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:05.263413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 975/1953] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:05.273536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:05.275171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:05.324788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:05.341661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be2fe-28ce-4003-aa5d-3ebaed3f9a74 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:05.346105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46030ccf-04f3-4f2b-9aae-5e9bd6b55532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:05.346767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46030ccf-04f3-4f2b-9aae-5e9bd6b55532 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:05.347055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46030ccf-04f3-4f2b-9aae-5e9bd6b55532 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:05.347334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46030ccf-04f3-4f2b-9aae-5e9bd6b55532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:05.361972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46030ccf-04f3-4f2b-9aae-5e9bd6b55532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:05.364537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46030ccf-04f3-4f2b-9aae-5e9bd6b55532 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:05.377247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 979/1954] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:06.185782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.187202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:06.200850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea507df7-e6ab-4d99-b417-219339e0c821 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.207501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.210786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.210786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.210786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] PATH_INFO: `/v3/projects/bccf2c43b8444d94bd8727cf3ad2730b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.210786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccf2c43b8444d94bd8727cf3ad2730b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:06.212639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:06.328664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccf2c43b8444d94bd8727cf3ad2730b', 'request_id': 'req-071edfd9-ea7d-437b-aef9-e98af18ec01f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:06.335774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccf2c43b8444d94bd8727cf3ad2730b', 'request_id': 'req-071edfd9-ea7d-437b-aef9-e98af18ec01f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:06.342242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccf2c43b8444d94bd8727cf3ad2730b', 'request_id': 'req-071edfd9-ea7d-437b-aef9-e98af18ec01f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:06.408926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] The token cache is being invalidate because project bccf2c43b8444d94bd8727cf3ad2730b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:06.409142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:06.409774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-071edfd9-ea7d-437b-aef9-e98af18ec01f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:06.411337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 976/1955] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:06 2022] DELETE /identity/v3/projects/bccf2c43b8444d94bd8727cf3ad2730b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:06.416326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.416846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.416955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.435800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:06.437047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:06.503952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 980/1956] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:06.511850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.512220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.512571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.532518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:06.534126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:06.542710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 977/1957] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:06.546979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.557675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce08e67-dd6b-40de-a0bb-bb6cdfda970c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.562954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.563652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.563943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.564251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.565658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:08:06.566042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:06.567692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:06.581239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 981/1958] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:08:06.585658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.597343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd57e4f-f1e4-4418-b4db-307bbeeaf586 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.602310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.602979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.603257 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.603592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.604288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:06.606209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:06.698185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 978/1959] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:06 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:06.702785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.713733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45e4d5-79c7-4ce8-afb4-3ba83a0c13c5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.771241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.771931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.772212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.772487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.773180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:06.775071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:06.901583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 982/1960] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:06 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:06.905848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.916588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a49bb-4fb9-466f-b05c-16aac90bb6a7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.921056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.921710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.921991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.922265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.922979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:06.924921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:06.935178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 979/1961] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:08:06.939285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.951786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b667f-4995-4c14-b68f-c7ad6277e5c8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.954947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be885cdf-410f-47ef-993b-a7d420888f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.956560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be885cdf-410f-47ef-993b-a7d420888f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.956849 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be885cdf-410f-47ef-993b-a7d420888f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.957130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be885cdf-410f-47ef-993b-a7d420888f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:06.957900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be885cdf-410f-47ef-993b-a7d420888f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:06.959526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be885cdf-410f-47ef-993b-a7d420888f6f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:06.969845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 983/1962] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:06.974289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:06.984597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b91315-a406-4d99-a784-9e0dffd28abf admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:06.989557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:06.990453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:06.990739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:06.991022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] PATH_INFO: `/v3/projects/d613e406953c470d98e991ba14968f6f/users/11fe4bf3cfdc416086b0efb5cf6f03b9/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:07.007254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d613e406953c470d98e991ba14968f6f, user_id=11fe4bf3cfdc416086b0efb5cf6f03b9, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:07.010071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:07.062015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 980/1963] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:06 2022] PUT /identity/v3/projects/d613e406953c470d98e991ba14968f6f/users/11fe4bf3cfdc416086b0efb5cf6f03b9/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:07.072718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c42449-e0a8-41de-b13e-3230acb65ff6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:07.073010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c42449-e0a8-41de-b13e-3230acb65ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:07.073281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c42449-e0a8-41de-b13e-3230acb65ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:07.115040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e1c42449-e0a8-41de-b13e-3230acb65ff6 None None] MFA Rules not processed for user `11fe4bf3cfdc416086b0efb5cf6f03b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:07.116640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e1c42449-e0a8-41de-b13e-3230acb65ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:07.165705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:07.201361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 984/1964] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:07.213996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:07.215819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:07.225906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:07.278046 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953bfe-14f4-41fb-9017-495bee31537c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:07.282281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0283a5-f38a-414e-bc5f-3fb623f4d9bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:07.282941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0283a5-f38a-414e-bc5f-3fb623f4d9bd service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:07.283271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0283a5-f38a-414e-bc5f-3fb623f4d9bd service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:07.283693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0283a5-f38a-414e-bc5f-3fb623f4d9bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:07.292242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0283a5-f38a-414e-bc5f-3fb623f4d9bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:07.293689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0283a5-f38a-414e-bc5f-3fb623f4d9bd service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:07.303337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 981/1965] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:16.764307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f2902-7311-4077-aeff-1f35d4b6bb18 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:16.764990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f2902-7311-4077-aeff-1f35d4b6bb18 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:16.764990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f2902-7311-4077-aeff-1f35d4b6bb18 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:16.766332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 985/1966] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:08:16.779964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:16.780444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:16.780535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:16.799706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:16.800948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:16.890087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 982/1967] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:16.910785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f2902-7311-4077-aeff-1f35d4b6bb18 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:16.913548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ff0f2902-7311-4077-aeff-1f35d4b6bb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:16.933175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f2902-7311-4077-aeff-1f35d4b6bb18 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:16.939232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:16.940051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:16.940440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:16.940824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:16.952445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:16.954007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:16.976557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 986/1968] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:16.984229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:16.988254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:17.064476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee7027-7ba9-4b8b-baf4-64d4d7e60c64 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:17.078194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:17.078194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:17.078194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:17.078194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] PATH_INFO: `/v3/users/9eac1fb288764b74baf30892237b3cdc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:17.078600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eac1fb288764b74baf30892237b3cdc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:17.078600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:17.466580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eac1fb288764b74baf30892237b3cdc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:17.467161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eac1fb288764b74baf30892237b3cdc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:17.468877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:17.544310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eac1fb288764b74baf30892237b3cdc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:17.558885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eac1fb288764b74baf30892237b3cdc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:17.573601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e13c1c47-30a4-4305-8a71-1d67020c4e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eac1fb288764b74baf30892237b3cdc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:17.580296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 983/1969] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:16 2022] DELETE /identity/v3/users/9eac1fb288764b74baf30892237b3cdc => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:17.819458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:17.831474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e29cf-001a-44d8-afea-229b09a31f56 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:17.892367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:17.893227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:17.893511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:17.893786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] PATH_INFO: `/v3/projects/285d8bae36f34e6a80a3982ebb097ea3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:17.894957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=285d8bae36f34e6a80a3982ebb097ea3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:17.897094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:18.151773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285d8bae36f34e6a80a3982ebb097ea3', 'request_id': 'req-85af986f-ffd6-485b-8d36-a82f93528bcc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:18.218160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285d8bae36f34e6a80a3982ebb097ea3', 'request_id': 'req-85af986f-ffd6-485b-8d36-a82f93528bcc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:18.225671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285d8bae36f34e6a80a3982ebb097ea3', 'request_id': 'req-85af986f-ffd6-485b-8d36-a82f93528bcc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:18.236512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] The token cache is being invalidate because project 285d8bae36f34e6a80a3982ebb097ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:18.236798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:18.237437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-85af986f-ffd6-485b-8d36-a82f93528bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:18.238963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 987/1970] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:17 2022] DELETE /identity/v3/projects/285d8bae36f34e6a80a3982ebb097ea3 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:18.250500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.250957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.251060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.269244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:18.270589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:18.340970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 984/1971] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:18.351062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.351062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.351062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.373334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:18.374570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:18.388103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 988/1972] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:08:18.388671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:18.404553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8a22-806b-49b8-b37e-ad95c9dde603 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:18.413542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:18.413542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.413542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.413542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.413945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:08:18.413945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:18.415451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:18.429031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 985/1973] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:08:18.433980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:18.445998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0c61f-de28-46c8-b38f-b67f43a71888 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:18.449123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:18.449854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.450143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.450423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.452531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:18.455191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:18.609337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 989/1974] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:18 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:18.613768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:18.624123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab5267-c7ba-4549-adbd-1ec7650be117 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:18.682650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:18.683301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.683632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.683915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.684606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:18.686581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:18.774508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 986/1975] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:18 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:18.779395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:18.794320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818b1da-c2db-4bd0-bf86-bce15fd2d150 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:18.799217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:18.799862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.800148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.800426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.801467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:18.803329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:18.819942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 990/1976] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:18.822000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:18.837225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1e75-651a-411e-bafe-10bfa65a7191 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:18.843999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236cf00-ea99-46e4-ac15-3568ba992848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:18.845335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236cf00-ea99-46e4-ac15-3568ba992848 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.845431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236cf00-ea99-46e4-ac15-3568ba992848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.845809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236cf00-ea99-46e4-ac15-3568ba992848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.846935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236cf00-ea99-46e4-ac15-3568ba992848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:18.849180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236cf00-ea99-46e4-ac15-3568ba992848 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:18.866109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 987/1977] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:08:18.869750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:18.879809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac025d10-5137-43a2-94ad-34422eb34cfe admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:18.885689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:18.886580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:18.886864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:18.887140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] PATH_INFO: `/v3/projects/41d45715613143379e732a9582f90cfe/users/6f02a227524644ce98542d8c5a00819c/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:18.899308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:18.903325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d45715613143379e732a9582f90cfe, user_id=6f02a227524644ce98542d8c5a00819c, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:18.905972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.030763 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 991/1978] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:18 2022] PUT /identity/v3/projects/41d45715613143379e732a9582f90cfe/users/6f02a227524644ce98542d8c5a00819c/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:19.042083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.042374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.042655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.071069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] MFA Rules not processed for user `6f02a227524644ce98542d8c5a00819c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:19.072306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:19.090604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:19.155808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 988/1979] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:19.161054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:19.234353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2859d9-5996-43e6-8948-88c6094213f2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:19.239101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454d013-fd2d-4609-9ed5-6ea9f93c676e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:19.239797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d013-fd2d-4609-9ed5-6ea9f93c676e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.240057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d013-fd2d-4609-9ed5-6ea9f93c676e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.240335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454d013-fd2d-4609-9ed5-6ea9f93c676e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.241664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d013-fd2d-4609-9ed5-6ea9f93c676e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:19.244211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454d013-fd2d-4609-9ed5-6ea9f93c676e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.413520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 992/1980] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:19 2022] POST /identity/v3/projects => generated 381 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:08:19.417439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:19.429718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf591d3-05ca-4364-86e8-506959b4b3b5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:19.478444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88633363-b3aa-444f-af6c-5c852690852c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.480731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88633363-b3aa-444f-af6c-5c852690852c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.480731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88633363-b3aa-444f-af6c-5c852690852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.498187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-88633363-b3aa-444f-af6c-5c852690852c None None] MFA Rules not processed for user `84114b0e2d6d4ea4b0b494d8b6e66a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:19.499599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-88633363-b3aa-444f-af6c-5c852690852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:19.514888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:19.518091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.518091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.518091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.518091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:19.520153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.589724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 993/1981] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:08:19.645021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 989/1982] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:19 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:19.649119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88633363-b3aa-444f-af6c-5c852690852c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:19.665331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88633363-b3aa-444f-af6c-5c852690852c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:19.670967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:19.673329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.673329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.673329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.673687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:19.675445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.688215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 994/1983] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:08:19.692637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:19.705689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba443-3560-4825-8afb-5517b9f01311 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:19.712400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:19.713399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.714368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.714739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] PATH_INFO: `/v3/projects/9250f7395bbb44a3839bb0a51820194d/users/cafb38604141408fbb1c25a8aeb13a6c/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.733020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9250f7395bbb44a3839bb0a51820194d, user_id=cafb38604141408fbb1c25a8aeb13a6c, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:19.735801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.813113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 990/1984] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:19 2022] PUT /identity/v3/projects/9250f7395bbb44a3839bb0a51820194d/users/cafb38604141408fbb1c25a8aeb13a6c/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:19.817245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:19.828328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e334ee-a606-4a34-91a8-6e3ba7669834 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:19.895530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beff511-4e4c-49e1-bb9f-f5de246ce90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:19.896223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beff511-4e4c-49e1-bb9f-f5de246ce90a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.896511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beff511-4e4c-49e1-bb9f-f5de246ce90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.896790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beff511-4e4c-49e1-bb9f-f5de246ce90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.898958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beff511-4e4c-49e1-bb9f-f5de246ce90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:19.901350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beff511-4e4c-49e1-bb9f-f5de246ce90a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.915231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 995/1985] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:08:19.920380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:19.940706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5dba2-ac9d-4694-bdd8-566543c9f22c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:19.946808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:19.948440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.948715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.949364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] PATH_INFO: `/v3/projects/9250f7395bbb44a3839bb0a51820194d/users/cafb38604141408fbb1c25a8aeb13a6c/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.952611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9250f7395bbb44a3839bb0a51820194d, user_id=cafb38604141408fbb1c25a8aeb13a6c, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:19.956225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:19.968117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 991/1986] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:19 2022] PUT /identity/v3/projects/9250f7395bbb44a3839bb0a51820194d/users/cafb38604141408fbb1c25a8aeb13a6c/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:19.978774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:19.979163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:19.979550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:19.986831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:20.007450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] MFA Rules not processed for user `cafb38604141408fbb1c25a8aeb13a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:20.008686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:20.092023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 996/1987] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:20.104582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:20.106884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:20.202573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50da4d9-3f8f-410d-ac9e-54a5aa6232c2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:20.207216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:20.208489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:20.208949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:20.215359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:20.222153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:20.223678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:20.234029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 992/1988] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:20.562499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:20.574826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95952a-0edf-472a-8332-ca9641d74cbc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:20.578685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17845537-c6de-4be4-81d8-377ee5b76589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:20.579335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845537-c6de-4be4-81d8-377ee5b76589 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:20.579648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845537-c6de-4be4-81d8-377ee5b76589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:20.579924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17845537-c6de-4be4-81d8-377ee5b76589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:20.638924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845537-c6de-4be4-81d8-377ee5b76589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:20.640430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17845537-c6de-4be4-81d8-377ee5b76589 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:20.649962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 997/1989] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:27.659285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:27.660841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:27.680039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:27.727668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e9cb1-119e-43c1-ac32-e25677c12e58 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:27.733361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:27.734238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:27.734525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:27.734804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] PATH_INFO: `/v3/users/5fe14dcfd2a4428aa90e5b49e0b9ece6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:27.736008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe14dcfd2a4428aa90e5b49e0b9ece6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:27.738012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:27.885162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe14dcfd2a4428aa90e5b49e0b9ece6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:27.886015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe14dcfd2a4428aa90e5b49e0b9ece6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:27.946589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe14dcfd2a4428aa90e5b49e0b9ece6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:27.954573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe14dcfd2a4428aa90e5b49e0b9ece6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:27.967725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe14dcfd2a4428aa90e5b49e0b9ece6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:27.973737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 993/1990] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:27 2022] DELETE /identity/v3/users/5fe14dcfd2a4428aa90e5b49e0b9ece6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:27.978374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:27.982251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:27.982251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:27.996990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:27.998248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:28.067174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 998/1991] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:08:28.074207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:28.075760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:28.132974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c9383-9870-43e1-97af-d2d7a216c9eb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:28.136682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:28.137596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:28.137878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:28.138155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:28.146992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:28.148440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:28.158080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 994/1992] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:28.632486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:28.633948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:28.646959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445dc44-3d87-4d74-9490-1713f0fa1714 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:28.652617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:28.653536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:28.653907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:28.654261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] PATH_INFO: `/v3/projects/8dc4d3956b7148ec9042560310d6532a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:28.655798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc4d3956b7148ec9042560310d6532a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:28.658801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:28.688740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:28.788400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc4d3956b7148ec9042560310d6532a', 'request_id': 'req-4c46e803-fdb9-4ac4-a85a-b950f608862a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:28.843753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc4d3956b7148ec9042560310d6532a', 'request_id': 'req-4c46e803-fdb9-4ac4-a85a-b950f608862a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:28.850434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc4d3956b7148ec9042560310d6532a', 'request_id': 'req-4c46e803-fdb9-4ac4-a85a-b950f608862a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:28.862372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] The token cache is being invalidate because project 8dc4d3956b7148ec9042560310d6532a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:28.862689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:28.863341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4c46e803-fdb9-4ac4-a85a-b950f608862a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:28.865322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 999/1993] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:28 2022] DELETE /identity/v3/projects/8dc4d3956b7148ec9042560310d6532a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:08:38.153895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:38.858718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:39.015337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:39.017646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:39.098877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48420d4-f561-4cdd-9ddc-8554d778c241 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:39.156820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:39.157805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:39.158093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:39.158380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] PATH_INFO: `/v3/users/6f02a227524644ce98542d8c5a00819c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:39.159635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f02a227524644ce98542d8c5a00819c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:39.161558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:39.361364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f02a227524644ce98542d8c5a00819c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:39.362099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f02a227524644ce98542d8c5a00819c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:39.440902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f02a227524644ce98542d8c5a00819c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:39.448342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f02a227524644ce98542d8c5a00819c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:39.460434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f02a227524644ce98542d8c5a00819c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:39.466473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 995/1994] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:39 2022] DELETE /identity/v3/users/6f02a227524644ce98542d8c5a00819c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:39.471148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:39.471600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:39.471893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:39.498765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:39.500079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:39.562931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1000/1995] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:39.573640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:39.575052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:39.606056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:39.691059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9810d4-c88c-4377-ade8-5a23f40b02c6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:39.694920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:39.697870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:39.697870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:39.697870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:39.707204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:39.708827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:39.718680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 996/1996] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:40.213986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:40.216829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:40.251851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a249cb-62cc-49c7-b693-3b6f231ff8e0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:40.258313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:40.259256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:40.259790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:40.260092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] PATH_INFO: `/v3/projects/41d45715613143379e732a9582f90cfe` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:40.261916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d45715613143379e732a9582f90cfe)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:40.263932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:40.373351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d45715613143379e732a9582f90cfe', 'request_id': 'req-18f0e69e-db47-4ded-8c40-632249b66f4a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.417688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d45715613143379e732a9582f90cfe', 'request_id': 'req-18f0e69e-db47-4ded-8c40-632249b66f4a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.425115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d45715613143379e732a9582f90cfe', 'request_id': 'req-18f0e69e-db47-4ded-8c40-632249b66f4a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.435727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] The token cache is being invalidate because project 41d45715613143379e732a9582f90cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:40.436014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.436655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.438223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1001/1997] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:40 2022] DELETE /identity/v3/projects/41d45715613143379e732a9582f90cfe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:40.442778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:40.452236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54388917-69c8-49e1-98d3-5fe10d29085a service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:40.509921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:40.510802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:40.511086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:40.511385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] PATH_INFO: `/v3/users/cafb38604141408fbb1c25a8aeb13a6c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:40.512577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafb38604141408fbb1c25a8aeb13a6c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:40.514542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:40.742074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafb38604141408fbb1c25a8aeb13a6c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.742719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafb38604141408fbb1c25a8aeb13a6c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.824328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafb38604141408fbb1c25a8aeb13a6c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.834255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafb38604141408fbb1c25a8aeb13a6c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.848936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27af2a0d-6f93-4d6c-889c-11a3edfe52b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafb38604141408fbb1c25a8aeb13a6c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:40.855660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 997/1998] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:40 2022] DELETE /identity/v3/users/cafb38604141408fbb1c25a8aeb13a6c => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:41.307850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:41.309592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:41.323095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0e69e-db47-4ded-8c40-632249b66f4a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:41.389076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:41.389953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:41.390240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:41.390518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] PATH_INFO: `/v3/projects/9250f7395bbb44a3839bb0a51820194d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:41.392700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9250f7395bbb44a3839bb0a51820194d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:41.394942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:41.417887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:41.526499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9250f7395bbb44a3839bb0a51820194d', 'request_id': 'req-1cb28828-956a-481c-87c2-030a7ec17b3b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:41.610399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9250f7395bbb44a3839bb0a51820194d', 'request_id': 'req-1cb28828-956a-481c-87c2-030a7ec17b3b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:41.618034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9250f7395bbb44a3839bb0a51820194d', 'request_id': 'req-1cb28828-956a-481c-87c2-030a7ec17b3b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:41.630090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] The token cache is being invalidate because project 9250f7395bbb44a3839bb0a51820194d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:41.632107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:41.632107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1cb28828-956a-481c-87c2-030a7ec17b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:41.632540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1002/1999] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:41 2022] DELETE /identity/v3/projects/9250f7395bbb44a3839bb0a51820194d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:08:41.638315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:41.638808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:41.638903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:41.644617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:41.676996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:41.678269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:41.740090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 998/2000] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:41.747998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:41.748295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:41.748579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:41.768455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:41.769298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:41.780162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1003/2001] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:41.784720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:41.794607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefebe0-eb71-4747-be70-2447d89b78dd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:41.800218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:41.800879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:41.801167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:41.801474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:41.802199 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:08:41.802510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:41.804153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:41.816452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 999/2002] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:08:41.820778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:41.830344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee927df-62de-415b-8f28-dc79cff66723 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:41.835097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:41.835780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:41.836072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:41.836384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:41.837110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:41.839007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:41.923993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1004/2003] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:41 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:41.928067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:41.938350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78662e0b-b734-4378-97fa-68329fabb730 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:41.999617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.000123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.000566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.000853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.001854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.003917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.093464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1000/2004] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:41 2022] POST /identity/v3/users => generated 301 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:42.098300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.108856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134c045-9490-4b92-8b72-17daa9199e01 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.114789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.115490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.115776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.116053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.116782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.118373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.139143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1005/2005] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:42.143935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.154149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5864fd-3799-4590-bc1c-42fdf61f7986 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.159439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d05254-9a6e-44cc-ab7c-ca89d322a666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.160126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d05254-9a6e-44cc-ab7c-ca89d322a666 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.160413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d05254-9a6e-44cc-ab7c-ca89d322a666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.160692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d05254-9a6e-44cc-ab7c-ca89d322a666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.161416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d05254-9a6e-44cc-ab7c-ca89d322a666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.163060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d05254-9a6e-44cc-ab7c-ca89d322a666 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.172978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1001/2006] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:42.177872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.187901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c48f-9736-4a72-a503-be38b8cd195e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.193552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.194550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.194830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.195102 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] PATH_INFO: `/v3/projects/cf6bae42a36644d2b275d978d289832b/users/0eeba867303444258a8795f76fc32a49/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.210033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6bae42a36644d2b275d978d289832b, user_id=0eeba867303444258a8795f76fc32a49, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.212691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.373088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1006/2007] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:42 2022] PUT /identity/v3/projects/cf6bae42a36644d2b275d978d289832b/users/0eeba867303444258a8795f76fc32a49/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:42.384690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.384987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.385271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.413380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] MFA Rules not processed for user `0eeba867303444258a8795f76fc32a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:42.414537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:42.460550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:42.482947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1002/2008] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:42.487685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.554804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c8c30-ae89-4e36-ae7a-7ef1e60e3ae2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.566049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.566049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.566049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.566049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.566388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.566388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.650258 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1007/2009] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:42 2022] POST /identity/v3/projects => generated 401 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:42.654435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.665392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93676a1-05a1-4182-9a13-bdda58bba976 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.724036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.724741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.725020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.725292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.725977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.727900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.811400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1003/2010] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:42 2022] POST /identity/v3/users => generated 307 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:42.816245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.826488 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4817dc-077b-44e7-a1fc-5e82ff5e0edc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.831816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.832482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.832768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.833046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.833772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.835353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.846019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1008/2011] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:42.850538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.860968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6adc1-7a7f-454f-89c9-3b8774c9d4d6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:42.865872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:42.866785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:42.867063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:42.867355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] PATH_INFO: `/v3/projects/618058a6685f45348324859b652f67d0/users/13b55fb00c4f4053952c5b45fc542ebd/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:42.882927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618058a6685f45348324859b652f67d0, user_id=13b55fb00c4f4053952c5b45fc542ebd, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:42.885587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:42.976994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1004/2012] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:42 2022] PUT /identity/v3/projects/618058a6685f45348324859b652f67d0/users/13b55fb00c4f4053952c5b45fc542ebd/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:42.980843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:42.991179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f60b4-f0d8-426f-bc8d-87d766602fe2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:43.004469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:43.053403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a3dbb-82f2-4ba1-976e-070a86278a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:43.054085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3dbb-82f2-4ba1-976e-070a86278a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:43.054363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3dbb-82f2-4ba1-976e-070a86278a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:43.054645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3dbb-82f2-4ba1-976e-070a86278a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:43.055496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3dbb-82f2-4ba1-976e-070a86278a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:43.057557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3dbb-82f2-4ba1-976e-070a86278a41 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:43.068974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1009/2013] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:43.073185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:43.085939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cd0a4-2f4a-458b-b076-8c0dd7257ed1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:43.092127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:43.092127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:43.092127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:43.094742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] PATH_INFO: `/v3/projects/618058a6685f45348324859b652f67d0/users/13b55fb00c4f4053952c5b45fc542ebd/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:43.094742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=618058a6685f45348324859b652f67d0, user_id=13b55fb00c4f4053952c5b45fc542ebd, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:43.097051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:43.108172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1005/2014] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:43 2022] PUT /identity/v3/projects/618058a6685f45348324859b652f67d0/users/13b55fb00c4f4053952c5b45fc542ebd/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:08:43.119406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:43.119707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:43.120012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:43.148124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] MFA Rules not processed for user `13b55fb00c4f4053952c5b45fc542ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:43.149380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:43.221152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1010/2015] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:08:43.232218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:43.233896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:43.293208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8206a-5181-4311-a058-9d3998742f4e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:43.296864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:43.297513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:43.297801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:43.298197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:43.308130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:43.309571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:43.319878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1006/2016] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:43.547026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:43.558447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c601179-350d-45cd-ba36-6b8e9f355655 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:43.561932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1412ceb-2e8d-4b73-aacd-ca741588f234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:43.562578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1412ceb-2e8d-4b73-aacd-ca741588f234 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:43.562866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1412ceb-2e8d-4b73-aacd-ca741588f234 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:43.563144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1412ceb-2e8d-4b73-aacd-ca741588f234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:43.620592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1412ceb-2e8d-4b73-aacd-ca741588f234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:43.622059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1412ceb-2e8d-4b73-aacd-ca741588f234 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:43.632190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1011/2017] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:52.970434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:53.015584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:53.037879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed18843-4657-49ae-8786-09d450035777 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:53.047493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:53.047493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:53.047493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:53.047493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] PATH_INFO: `/v3/users/0eeba867303444258a8795f76fc32a49` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:53.047843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eeba867303444258a8795f76fc32a49)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:53.047843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:53.270544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eeba867303444258a8795f76fc32a49'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:53.271308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eeba867303444258a8795f76fc32a49'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:53.346879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eeba867303444258a8795f76fc32a49'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:53.355322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eeba867303444258a8795f76fc32a49'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:53.367778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eeba867303444258a8795f76fc32a49'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:53.373646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1007/2018] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:52 2022] DELETE /identity/v3/users/0eeba867303444258a8795f76fc32a49 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:53.378181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:53.378674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:53.378756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:53.396699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:53.397946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:53.431778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:53.458100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1012/2019] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:53.470019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:53.471525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:53.529699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72d795-ef21-411a-b7b6-ed09e6b3895b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:53.533284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:53.533936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:53.534223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:53.534505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:53.544658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:53.546089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:53.555772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1008/2020] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:08:53.969805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:53.971435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:53.984935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe48cf-054b-41d3-b243-ccce80fa1bc6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:53.990933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:53.991845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:54.001120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:54.001120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] PATH_INFO: `/v3/projects/cf6bae42a36644d2b275d978d289832b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:54.001120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6bae42a36644d2b275d978d289832b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:54.001120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:54.102915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6bae42a36644d2b275d978d289832b', 'request_id': 'req-8b732f33-a45c-4f22-b315-f782f9a742cc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.173397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6bae42a36644d2b275d978d289832b', 'request_id': 'req-8b732f33-a45c-4f22-b315-f782f9a742cc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.180966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6bae42a36644d2b275d978d289832b', 'request_id': 'req-8b732f33-a45c-4f22-b315-f782f9a742cc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.191681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] The token cache is being invalidate because project cf6bae42a36644d2b275d978d289832b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:54.191992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.192730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.194393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1013/2021] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:53 2022] DELETE /identity/v3/projects/cf6bae42a36644d2b275d978d289832b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:54.198351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:54.216744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae41900-39fb-4f98-ac66-8ce0dc727de9 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:54.294928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:54.295860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:54.296142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:54.296418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] PATH_INFO: `/v3/users/13b55fb00c4f4053952c5b45fc542ebd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:54.297667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b55fb00c4f4053952c5b45fc542ebd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:54.299794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:54.305306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:54.551122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b55fb00c4f4053952c5b45fc542ebd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.552091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b55fb00c4f4053952c5b45fc542ebd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.661877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b55fb00c4f4053952c5b45fc542ebd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.669606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b55fb00c4f4053952c5b45fc542ebd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.685413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6c93c3ec-df04-46d6-a670-059908ce17c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b55fb00c4f4053952c5b45fc542ebd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:54.693855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1009/2022] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:08:54 2022] DELETE /identity/v3/users/13b55fb00c4f4053952c5b45fc542ebd => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:55.284652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:55.286262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:55.300648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b732f33-a45c-4f22-b315-f782f9a742cc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:55.376740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:55.377607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:55.377891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:55.378194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] PATH_INFO: `/v3/projects/618058a6685f45348324859b652f67d0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:55.379511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618058a6685f45348324859b652f67d0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:55.382078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:55.506669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618058a6685f45348324859b652f67d0', 'request_id': 'req-be456f2a-cba8-4392-b6e1-acd7a33a4675'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:55.633319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618058a6685f45348324859b652f67d0', 'request_id': 'req-be456f2a-cba8-4392-b6e1-acd7a33a4675'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:55.641050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618058a6685f45348324859b652f67d0', 'request_id': 'req-be456f2a-cba8-4392-b6e1-acd7a33a4675'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:55.643241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:55.653100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] The token cache is being invalidate because project 618058a6685f45348324859b652f67d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:08:55.653388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:55.654029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-be456f2a-cba8-4392-b6e1-acd7a33a4675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:08:55.655599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1014/2023] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:08:55 2022] DELETE /identity/v3/projects/618058a6685f45348324859b652f67d0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:55.666907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:55.666907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:55.666907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:55.700197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:55.701419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:55.762556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1010/2024] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:55.770536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:55.770827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:55.771140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:55.791521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:55.792742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:55.800940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1015/2025] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:55.805618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:55.816248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6de03-04c7-4cef-8be2-25824dfc936d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:55.821960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:55.822639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:55.822929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:55.823209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:55.823971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:08:55.824286 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:55.825905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:55.841744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1011/2026] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:08:55.842959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:55.853411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9416f9-4ac1-4df9-8463-bcee707968ea None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:55.858187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:55.867582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:55.867582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:55.867582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:55.867582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:55.867582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:55.961924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1016/2027] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:55 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:55.966141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:55.976967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89047f-f947-4f53-9f5c-d34ca2961720 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:56.035654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:56.036322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:56.037708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:56.038020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:56.038713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:56.040729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:56.048395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:56.154363 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1012/2028] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:55 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:56.155095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:56.167250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec4862-5d83-4c34-a232-bd96917e8fa6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:56.171566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:56.172361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:56.172677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:56.172951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:56.173678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:56.175618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:56.187458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1017/2029] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:56.191552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:56.203544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c7190-8a66-4439-afe5-d8d42f2d55f9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:56.209252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dee14-512b-4f97-96b3-946f5b9e3ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:56.209939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dee14-512b-4f97-96b3-946f5b9e3ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:56.210224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dee14-512b-4f97-96b3-946f5b9e3ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:56.210504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dee14-512b-4f97-96b3-946f5b9e3ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:56.211233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dee14-512b-4f97-96b3-946f5b9e3ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:56.212863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dee14-512b-4f97-96b3-946f5b9e3ab0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:56.224346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1013/2030] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:56.228610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:56.238617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccaaae-4b84-4538-89b4-911e8fd3c12f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:56.243908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:56.244817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:56.245099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:56.245400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] PATH_INFO: `/v3/projects/829e4ac9d1fb4f318b0b3b3b0562e27f/users/e730e55e67524766b113e4c8ad97bb00/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:56.261586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=829e4ac9d1fb4f318b0b3b3b0562e27f, user_id=e730e55e67524766b113e4c8ad97bb00, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:56.264292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:56.371196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1018/2031] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:56 2022] PUT /identity/v3/projects/829e4ac9d1fb4f318b0b3b3b0562e27f/users/e730e55e67524766b113e4c8ad97bb00/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:56.376784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879156-a665-411c-935c-07bd43f00020 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:56.377074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879156-a665-411c-935c-07bd43f00020 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:56.377355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879156-a665-411c-935c-07bd43f00020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:56.401240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-09879156-a665-411c-935c-07bd43f00020 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:56.402503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09879156-a665-411c-935c-07bd43f00020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:56.463733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1014/2032] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:56.473260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:56.475034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:56.542190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7311274-11f0-4baa-a205-95f33b8e1bbd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:56.545946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:56.546596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:56.546884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:56.547165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:56.556444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:56.558041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:56.567579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1019/2033] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:08:57.961328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:57.961891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:57.961891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:57.990791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] MFA Rules not processed for user `e730e55e67524766b113e4c8ad97bb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:08:57.992203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:08:58.059528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1015/2034] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:08:58.069745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:58.080052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39527b67-b92b-46c5-9c17-cdef3880f843 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:58.087020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:58.087709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:58.087997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:58.088588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:58.089289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:58.091595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:58.178897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1020/2035] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:08:58 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:58.183700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:58.199101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc44281-54a9-451a-a826-de0ceb1f81be None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:58.273013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:58.273727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:58.274012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:58.274290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:58.275002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:58.276997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:58.422463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:58.430251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1016/2036] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:08:58 2022] POST /identity/v3/users => generated 292 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:08:58.439985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:58.448745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80c1cd-19cb-4db9-befa-c884cb58fc97 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:58.454269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:58.455310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:58.455715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:58.456065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:58.457021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:58.458787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:58.477931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1021/2037] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:58.479390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:58.490124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c7b64-81c0-4d7e-99c7-037d60c78afd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:58.495202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:58.496208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:58.496501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:58.496783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] PATH_INFO: `/v3/projects/7206d81fa5de492696a94b34ed56faaf/users/eeec8c2626e94409881ccc0acccf59fd/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:58.512536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7206d81fa5de492696a94b34ed56faaf, user_id=eeec8c2626e94409881ccc0acccf59fd, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:58.515189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:58.657070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1017/2038] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:58 2022] PUT /identity/v3/projects/7206d81fa5de492696a94b34ed56faaf/users/eeec8c2626e94409881ccc0acccf59fd/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:58.666392 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:58.676585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cea3c0-4223-4f5e-81ad-6c2faeac9834 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:58.733624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:08:58.739545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83589447-c0a7-4d0e-bbb7-478da34f2033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:58.740845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83589447-c0a7-4d0e-bbb7-478da34f2033 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:58.741550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83589447-c0a7-4d0e-bbb7-478da34f2033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:58.741550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83589447-c0a7-4d0e-bbb7-478da34f2033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:58.742156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83589447-c0a7-4d0e-bbb7-478da34f2033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:58.743779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83589447-c0a7-4d0e-bbb7-478da34f2033 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:58.753775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1022/2039] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:08:58.758823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:08:58.769565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c0905-6a5f-4bf3-a0b4-fbc16955e7d1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:08:58.775243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a0bf5-d08c-4641-858f-6de6af12db17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:08:58.776169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a0bf5-d08c-4641-858f-6de6af12db17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:58.776461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a0bf5-d08c-4641-858f-6de6af12db17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:58.776745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a0bf5-d08c-4641-858f-6de6af12db17 admin admin] PATH_INFO: `/v3/projects/7206d81fa5de492696a94b34ed56faaf/users/eeec8c2626e94409881ccc0acccf59fd/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:08:58.778740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a0bf5-d08c-4641-858f-6de6af12db17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7206d81fa5de492696a94b34ed56faaf, user_id=eeec8c2626e94409881ccc0acccf59fd, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:08:58.781543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a0bf5-d08c-4641-858f-6de6af12db17 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:08:58.792496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1018/2040] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:08:58 2022] PUT /identity/v3/projects/7206d81fa5de492696a94b34ed56faaf/users/eeec8c2626e94409881ccc0acccf59fd/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:08:59.984972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:08:59.985464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:08:59.985557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:00.003777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] MFA Rules not processed for user `70d6f15465b84182a28d96d1808a9660`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:00.004980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:00.063004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1023/2041] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:08:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:00.212920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99335fdd-456a-4521-b47b-190f6f335af5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:00.213482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99335fdd-456a-4521-b47b-190f6f335af5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:00.213482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99335fdd-456a-4521-b47b-190f6f335af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:00.244091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-99335fdd-456a-4521-b47b-190f6f335af5 None None] MFA Rules not processed for user `eeec8c2626e94409881ccc0acccf59fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:00.245616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99335fdd-456a-4521-b47b-190f6f335af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:00.325774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1019/2042] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:00.338668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:00.340336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:00.404435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17412fe-b5ff-49ab-819c-a68b3a483192 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:00.408149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:00.408883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:00.409250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:00.409619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:00.470393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:00.471892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:00.482437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1024/2043] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:09:08.201917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:08.202403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:08.202712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:08.233180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:08.234455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:08.304468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1020/2044] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:09:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:08.354329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:08.356360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:08.435722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:08.465400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61153e-0f0f-47e6-94f5-9c7360baff92 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:08.470829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:08.471663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:08.471873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:08.472174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:08.481566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:08.483217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:08.499936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1025/2045] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:09:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:09:08.626446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:08.631784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:08.692427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762835c8-9c3f-43e4-8068-97cd0f18a46c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:08.702696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2bf16-f64b-4456-afdf-93d00a842449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:08.703741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2bf16-f64b-4456-afdf-93d00a842449 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:08.704049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2bf16-f64b-4456-afdf-93d00a842449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:08.704335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2bf16-f64b-4456-afdf-93d00a842449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:08.723283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2bf16-f64b-4456-afdf-93d00a842449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:08.725013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2bf16-f64b-4456-afdf-93d00a842449 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:08.738672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1021/2046] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:15.087214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:15.091179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:15.295936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a10c5-ae3f-4280-b6f4-b681501f41b7 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:15.301980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:15.303034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:15.303330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:15.303647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] PATH_INFO: `/v3/users/11fe4bf3cfdc416086b0efb5cf6f03b9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:15.305275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fe4bf3cfdc416086b0efb5cf6f03b9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:15.308566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:15.553938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fe4bf3cfdc416086b0efb5cf6f03b9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:15.570125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fe4bf3cfdc416086b0efb5cf6f03b9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:15.586913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fe4bf3cfdc416086b0efb5cf6f03b9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:15.700952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fe4bf3cfdc416086b0efb5cf6f03b9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:15.713527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fe4bf3cfdc416086b0efb5cf6f03b9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:15.735507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1026/2047] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:15 2022] DELETE /identity/v3/users/11fe4bf3cfdc416086b0efb5cf6f03b9 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:15.740669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:15.741331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:15.741459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:15.749050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:15.782518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:15.783937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:15.881233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1022/2048] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:15.891211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:15.892755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:15.955506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11efb-0b1d-4ca0-862a-e969dcb45e5d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:15.959024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552dab-81f7-4cc6-9566-04f4544d18fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:15.959720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552dab-81f7-4cc6-9566-04f4544d18fd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:15.960003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552dab-81f7-4cc6-9566-04f4544d18fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:15.960281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552dab-81f7-4cc6-9566-04f4544d18fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:15.971158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552dab-81f7-4cc6-9566-04f4544d18fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:15.982516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552dab-81f7-4cc6-9566-04f4544d18fd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:15.991502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1027/2049] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:16.621209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:16.622963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:16.651391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2b4d0-765a-4916-abb8-c0d2f0356bbf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:16.658230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:16.659185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:16.659583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:16.659953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] PATH_INFO: `/v3/projects/d613e406953c470d98e991ba14968f6f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:16.661237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d613e406953c470d98e991ba14968f6f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:16.663311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:16.884230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd613e406953c470d98e991ba14968f6f', 'request_id': 'req-541d9ca7-8547-4b2a-b501-4d11cad9839f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:16.900171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd613e406953c470d98e991ba14968f6f', 'request_id': 'req-541d9ca7-8547-4b2a-b501-4d11cad9839f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:16.910202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd613e406953c470d98e991ba14968f6f', 'request_id': 'req-541d9ca7-8547-4b2a-b501-4d11cad9839f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:17.010016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] The token cache is being invalidate because project d613e406953c470d98e991ba14968f6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:17.011654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:17.012438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-541d9ca7-8547-4b2a-b501-4d11cad9839f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:17.014681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1023/2050] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:16 2022] DELETE /identity/v3/projects/d613e406953c470d98e991ba14968f6f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:17.021515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.022091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.022091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.034045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:17.050846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:17.052606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:17.162490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1028/2051] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:17.168162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.168581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.168973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.223340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:17.227522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:17.250067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1024/2052] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:17.255635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:17.273918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7851a7fc-b358-43b1-9723-da6e968ee6d3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:17.280726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:17.281762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.282492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.283234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.284727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:09:17.285236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:17.287091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:17.312106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1029/2053] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:09:17.313483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:17.324910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e66f0-567c-4fb5-9f68-744c8d418812 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:17.330281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:17.331237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.331565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.331848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.332569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:17.335147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:17.458104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1025/2054] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:17 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:17.464360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:17.481325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a66de-e84c-43e8-b7b0-f8b1d5d5ed87 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:17.635661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:17.644328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.644328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.644328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.644328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:17.644328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:17.836320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1030/2055] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:17 2022] POST /identity/v3/users => generated 287 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:17.840390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:17.855966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22fcd-5807-46b8-a5b9-0ad767d96fef admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:17.861459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:17.862703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.863221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.863916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.864880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:17.866967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:17.881013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1026/2056] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:17.884780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:17.896898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c42da-cda2-4396-ba1a-4cfeea181e81 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:17.904024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71305b-dca9-4d9b-a335-99982bd3e154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:17.904778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71305b-dca9-4d9b-a335-99982bd3e154 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.905063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71305b-dca9-4d9b-a335-99982bd3e154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.905343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71305b-dca9-4d9b-a335-99982bd3e154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.906128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71305b-dca9-4d9b-a335-99982bd3e154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:17.907930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71305b-dca9-4d9b-a335-99982bd3e154 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:17.928900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1031/2057] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:09:17.934220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:17.947791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad183aed-81ae-4819-b2fb-75ba18338793 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:17.953791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:17.955067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:17.955655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:17.956115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] PATH_INFO: `/v3/projects/bdb225966010445785ba6d65c068176c/users/2311f0b1e7074500a6964504a7aee218/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:17.974441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb225966010445785ba6d65c068176c, user_id=2311f0b1e7074500a6964504a7aee218, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:17.976651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:18.051415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1027/2058] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:17 2022] PUT /identity/v3/projects/bdb225966010445785ba6d65c068176c/users/2311f0b1e7074500a6964504a7aee218/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:18.057423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0a5e9-581d-4727-b797-3cb5d3147725 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:18.057932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0a5e9-581d-4727-b797-3cb5d3147725 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:18.058210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0a5e9-581d-4727-b797-3cb5d3147725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:18.102010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08c0a5e9-581d-4727-b797-3cb5d3147725 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:18.102696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08c0a5e9-581d-4727-b797-3cb5d3147725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:18.256123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1032/2059] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:18.269344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:18.276568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:18.314550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:18.376599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337719-69b3-43e3-833d-7b6adecf8d4a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:18.385893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:18.388348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:18.389028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:18.389511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:18.404683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:18.406295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:18.419675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1028/2060] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:27.350897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:27.351591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:27.351591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:27.383992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:27.385803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] MFA Rules not processed for user `2311f0b1e7074500a6964504a7aee218`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:27.387868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:27.474479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1033/2061] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:27.478750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:27.501512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c732270-05e0-4940-a092-aa989e06ad4c service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:27.508762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:27.510037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:27.510667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:27.511095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:27.512079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:27.515097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:27.702086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1029/2062] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:27 2022] POST /identity/v3/projects => generated 371 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:27.705686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:27.723902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47b04-a6ec-4786-99a5-a88cb5ead2b8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:27.879511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:27.881744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:27.882340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:27.882914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:27.884452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:27.889173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:27.995830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1034/2063] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:27 2022] POST /identity/v3/users => generated 292 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:28.000746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:28.032094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2b893-c5b2-4ac3-8035-317ad87cd09b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:28.037322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:28.038151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:28.038530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:28.038665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:28.039424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:28.041055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:28.074924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1030/2064] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:28.076287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:28.087134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcad09-278c-46c1-b9d6-942051d8a2ec admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:28.093150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:28.094230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:28.094529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:28.094795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] PATH_INFO: `/v3/projects/fdb92af186e84f6eb89fb15d292b86ed/users/ec1683d81b244667bf39bb2a53c73838/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:28.127529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb92af186e84f6eb89fb15d292b86ed, user_id=ec1683d81b244667bf39bb2a53c73838, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:28.130373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:28.263260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1035/2065] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:28 2022] PUT /identity/v3/projects/fdb92af186e84f6eb89fb15d292b86ed/users/ec1683d81b244667bf39bb2a53c73838/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:28.267557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:28.281062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d81a5-4863-4211-93f3-63b033195f73 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:28.368599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293614e-65fa-4c8f-8d99-946b66065cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:28.369331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293614e-65fa-4c8f-8d99-946b66065cde admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:28.369595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293614e-65fa-4c8f-8d99-946b66065cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:28.369871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293614e-65fa-4c8f-8d99-946b66065cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:28.370641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293614e-65fa-4c8f-8d99-946b66065cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:28.372393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293614e-65fa-4c8f-8d99-946b66065cde admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:28.384679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1031/2066] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:28.389433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:28.402870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945a03c-35d1-44dc-887e-aba00a461693 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:28.409270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:28.410334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:28.410724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:28.411110 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] PATH_INFO: `/v3/projects/fdb92af186e84f6eb89fb15d292b86ed/users/ec1683d81b244667bf39bb2a53c73838/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:28.413365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb92af186e84f6eb89fb15d292b86ed, user_id=ec1683d81b244667bf39bb2a53c73838, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:28.416277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:28.428935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1036/2067] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:28 2022] PUT /identity/v3/projects/fdb92af186e84f6eb89fb15d292b86ed/users/ec1683d81b244667bf39bb2a53c73838/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:31.993077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712e194-37f9-4edb-8bc5-8778296ac15a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:31.994370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712e194-37f9-4edb-8bc5-8778296ac15a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:31.994835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712e194-37f9-4edb-8bc5-8778296ac15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:32.066896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7712e194-37f9-4edb-8bc5-8778296ac15a None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:32.068202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7712e194-37f9-4edb-8bc5-8778296ac15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:32.163121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:32.164891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1032/2068] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jun 16 18:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:32.179246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:32.181201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:32.254813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:32.292765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a8b2c-c049-41f1-9b3d-1a6b17007d0b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:32.299685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736be6b4-450d-4c5e-8ce7-3e729d6ddcbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:32.300911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736be6b4-450d-4c5e-8ce7-3e729d6ddcbf service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:32.301180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736be6b4-450d-4c5e-8ce7-3e729d6ddcbf service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:32.301462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736be6b4-450d-4c5e-8ce7-3e729d6ddcbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:32.310596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736be6b4-450d-4c5e-8ce7-3e729d6ddcbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:32.312349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736be6b4-450d-4c5e-8ce7-3e729d6ddcbf service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:32.322036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1037/2069] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:09:33.210734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223571-a94b-49ae-81fc-039ca4a29724 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:33.211326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223571-a94b-49ae-81fc-039ca4a29724 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:33.211642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb223571-a94b-49ae-81fc-039ca4a29724 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:33.214041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1033/2070] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:09:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:09:33.225272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:33.226381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:33.226381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:33.315504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] MFA Rules not processed for user `efaf16b0733b426da91c5f5ae08deaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:33.316803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:33.458855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb223571-a94b-49ae-81fc-039ca4a29724 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:33.464681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cb223571-a94b-49ae-81fc-039ca4a29724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:33.580547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1038/2071] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:09:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:33.598198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:33.598198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:33.605499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb223571-a94b-49ae-81fc-039ca4a29724 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:33.612533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:33.613569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:33.613793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fbbe8-7633-427a-a1fe-65e11af9603f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:33.613980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:33.614324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] PATH_INFO: `/v3/users/e730e55e67524766b113e4c8ad97bb00` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:33.616812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e730e55e67524766b113e4c8ad97bb00)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:33.619211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:33.619987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:33.620694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:33.621009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:33.621290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:33.650284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:33.651946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:33.683185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1039/2072] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:33.894711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e730e55e67524766b113e4c8ad97bb00'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:33.895570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e730e55e67524766b113e4c8ad97bb00'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:33.993959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e730e55e67524766b113e4c8ad97bb00'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.004855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e730e55e67524766b113e4c8ad97bb00'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.023318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e730e55e67524766b113e4c8ad97bb00'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.032955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1034/2073] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:33 2022] DELETE /identity/v3/users/e730e55e67524766b113e4c8ad97bb00 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:34.333243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:34.358598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3d9f-9dc5-4254-ac5c-055720d067c5 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:34.444864 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:34.445724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:34.446189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:34.446521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] PATH_INFO: `/v3/projects/829e4ac9d1fb4f318b0b3b3b0562e27f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:34.448197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=829e4ac9d1fb4f318b0b3b3b0562e27f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:34.450818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:34.473979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:34.624767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '829e4ac9d1fb4f318b0b3b3b0562e27f', 'request_id': 'req-0df4c9fc-5922-4113-8580-b113c2a8c955'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.694625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '829e4ac9d1fb4f318b0b3b3b0562e27f', 'request_id': 'req-0df4c9fc-5922-4113-8580-b113c2a8c955'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.703696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '829e4ac9d1fb4f318b0b3b3b0562e27f', 'request_id': 'req-0df4c9fc-5922-4113-8580-b113c2a8c955'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.719272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] The token cache is being invalidate because project 829e4ac9d1fb4f318b0b3b3b0562e27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:34.719876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.721734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:34.726521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1040/2074] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:34 2022] DELETE /identity/v3/projects/829e4ac9d1fb4f318b0b3b3b0562e27f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:34.741586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:34.743786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:34.761827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06451-28db-47ee-b2d7-d05ddd547151 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:34.848789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:34.849931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:34.850675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:34.850789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] PATH_INFO: `/v3/users/eeec8c2626e94409881ccc0acccf59fd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:34.852331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeec8c2626e94409881ccc0acccf59fd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:34.854333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:34.886159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:35.119953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeec8c2626e94409881ccc0acccf59fd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:35.120721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeec8c2626e94409881ccc0acccf59fd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:35.224099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeec8c2626e94409881ccc0acccf59fd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:35.232322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeec8c2626e94409881ccc0acccf59fd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:35.251709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-af45abf1-9b66-42ed-8711-9cf9ba61557c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeec8c2626e94409881ccc0acccf59fd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:35.262629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1035/2075] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:34 2022] DELETE /identity/v3/users/eeec8c2626e94409881ccc0acccf59fd => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:36.029794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:36.032082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.063389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.063900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.064085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.064985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4c9fc-5922-4113-8580-b113c2a8c955 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:36.096104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] MFA Rules not processed for user `ec1683d81b244667bf39bb2a53c73838`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:36.097708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.141801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:36.143552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.143784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.144621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] PATH_INFO: `/v3/projects/7206d81fa5de492696a94b34ed56faaf` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.146229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7206d81fa5de492696a94b34ed56faaf)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:36.149263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:36.199187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1036/2076] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:36.223581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:36.225307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.290487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7206d81fa5de492696a94b34ed56faaf', 'request_id': 'req-a48444c8-348a-4ec8-983d-77f61e15a64d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.333760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:36.338653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8864f-011b-4fe6-93bc-86a91e23bb72 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:36.383893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7206d81fa5de492696a94b34ed56faaf', 'request_id': 'req-a48444c8-348a-4ec8-983d-77f61e15a64d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.392367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7206d81fa5de492696a94b34ed56faaf', 'request_id': 'req-a48444c8-348a-4ec8-983d-77f61e15a64d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.404486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] The token cache is being invalidate because project 7206d81fa5de492696a94b34ed56faaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:36.404779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.406175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.407198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1041/2077] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:36 2022] DELETE /identity/v3/projects/7206d81fa5de492696a94b34ed56faaf => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:36.411174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:36.413123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.414900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:36.415693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.415982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.416264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.418024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.452384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:36.481641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:36.483088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48444c8-348a-4ec8-983d-77f61e15a64d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:36.483408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda626a9-e55a-44bc-8f01-9a56fae793ac service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:36.489443 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:36.490322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.490603 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.490904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] PATH_INFO: `/v3/users/84114b0e2d6d4ea4b0b494d8b6e66a0e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.492350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=84114b0e2d6d4ea4b0b494d8b6e66a0e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:36.493977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1037/2078] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:36.494306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:36.497309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cdd7c-245d-4bd4-81c1-48dcc05dbe93 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.497962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cdd7c-245d-4bd4-81c1-48dcc05dbe93 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.498251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cdd7c-245d-4bd4-81c1-48dcc05dbe93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.518050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a52cdd7c-245d-4bd4-81c1-48dcc05dbe93 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:36.519291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a52cdd7c-245d-4bd4-81c1-48dcc05dbe93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.536552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1038/2079] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:36.538324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.538616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.538900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.559120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:36.560459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:36.568814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1039/2080] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:36.573325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:36.584054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce369bc0-2354-4f28-8bc4-28a22418150e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:36.589042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:36.589725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.590049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.590282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.591449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:09:36.591449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:36.593650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:36.617593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1040/2081] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:09:36.618764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:36.631584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23110374-c9ee-4ae8-85fc-e02ed4d0613e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:36.637337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:36.638002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.638585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.638585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.639249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:36.641798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:36.773723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1041/2082] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:36 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:36.778144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:36.846559 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84114b0e2d6d4ea4b0b494d8b6e66a0e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.848333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6bd80-4738-476a-a26e-2f3ab6fea93e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:36.853331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:36.854025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:36.854308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:36.854624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:36.855419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:36.857428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:36.860249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84114b0e2d6d4ea4b0b494d8b6e66a0e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.870227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84114b0e2d6d4ea4b0b494d8b6e66a0e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:36.983698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1042/2083] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:36 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:36.990946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.002856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ff8a4-3e17-4014-a2f3-1dbcb642bae8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.007923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.008594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.008909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.009182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.009907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.011519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.019005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84114b0e2d6d4ea4b0b494d8b6e66a0e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.019251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:37.019707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84114b0e2d6d4ea4b0b494d8b6e66a0e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.024560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1043/2084] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:09:37.027032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1042/2085] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:36 2022] DELETE /identity/v3/users/84114b0e2d6d4ea4b0b494d8b6e66a0e => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:37.034364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.036168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:37.050276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc34656-7c74-41d6-8b2f-701e372483c2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.116181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abb957-4407-45b2-baac-26606730e781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.117115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abb957-4407-45b2-baac-26606730e781 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.117115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abb957-4407-45b2-baac-26606730e781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.117414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abb957-4407-45b2-baac-26606730e781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.118152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abb957-4407-45b2-baac-26606730e781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.119811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abb957-4407-45b2-baac-26606730e781 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.131303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1044/2086] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:37.135710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.147990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d848b2-9e49-4ede-9eef-eb80a89684ab admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.163525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.164444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.164897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.165173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] PATH_INFO: `/v3/projects/495f0db42be04e6dbf79141ab7fe8a1c/users/cbd10fb86c464c4c8265726d3f27203c/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.184829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495f0db42be04e6dbf79141ab7fe8a1c, user_id=cbd10fb86c464c4c8265726d3f27203c, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.188396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.294403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1043/2087] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:37 2022] PUT /identity/v3/projects/495f0db42be04e6dbf79141ab7fe8a1c/users/cbd10fb86c464c4c8265726d3f27203c/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:37.305280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.305510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.305790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.338376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] MFA Rules not processed for user `cbd10fb86c464c4c8265726d3f27203c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:37.339688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:37.354983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.357819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:37.434651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1045/2088] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:37.440221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.454816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cac78-36d2-4066-a6ad-468872bbad17 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.463029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.464433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.464795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.465157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] PATH_INFO: `/v3/projects/7385d827292c44cf91ca0689dfc3753a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.467293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7385d827292c44cf91ca0689dfc3753a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.469518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.514369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:37.520948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18563fc-3640-4b10-8af4-78df4928526f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.525758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.526465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.526747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.527185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.527944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.530150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.602112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:37.623293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7385d827292c44cf91ca0689dfc3753a', 'request_id': 'req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.635849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1046/2089] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:37 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:37.648653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.727524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7385d827292c44cf91ca0689dfc3753a', 'request_id': 'req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.736400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7385d827292c44cf91ca0689dfc3753a', 'request_id': 'req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.749761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] The token cache is being invalidate because project 7385d827292c44cf91ca0689dfc3753a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:37.750171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.751853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c5d51e4a-eba7-487c-b5d4-0ccae83d7857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:37.753082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1044/2090] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:37 2022] DELETE /identity/v3/projects/7385d827292c44cf91ca0689dfc3753a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:09:37.756479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea7f49-1198-4b2f-b470-ca9baf43cd84 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.759266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb45aca-f9ce-43b3-a8bd-80e9ba592586 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.759728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb45aca-f9ce-43b3-a8bd-80e9ba592586 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.760227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb45aca-f9ce-43b3-a8bd-80e9ba592586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.762671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.763783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.771166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.771166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.771166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.771166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.785325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6cb45aca-f9ce-43b3-a8bd-80e9ba592586 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:37.786569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6cb45aca-f9ce-43b3-a8bd-80e9ba592586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:37.795056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1045/2091] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:37.803805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.804091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.804370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.823459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:37.824746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:37.833426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1046/2092] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:37.837719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.848171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e63c3-4691-40c4-847d-f364734acdf4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.856132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.856819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.857106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.857385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.858132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:09:37.858441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.860727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.873563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1047/2093] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:09:37.878337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.888942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17830d3c-068a-4d80-ad75-b66ee9648670 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.897407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.897407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.897407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.897407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.897841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.897932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.938289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1047/2094] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:37 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:37.939591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:37.941643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:37.955383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133a1d-a7de-4b71-9187-368f0e678ea0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:37.971967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:37.973225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:37.973706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:37.974177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:37.975522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:37.979534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:37.994826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1048/2095] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:09:38.001458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.010625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1048/2096] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:37 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:38.016124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.019575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa6f71-7de2-44a9-9047-e910474833a0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.088575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.089561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.090300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.090733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.094474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.096841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.097732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e7037-a42b-4098-b738-3057fc78b227 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.103143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.103907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.104237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.104665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.105394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.107438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.108802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1049/2097] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:09:38.113284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.123222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1f32d-9089-4430-8f16-440236afe3e9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.128263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4b201-bfa1-4f05-86f8-3773f8214dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.130549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4b201-bfa1-4f05-86f8-3773f8214dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.130549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4b201-bfa1-4f05-86f8-3773f8214dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.130549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4b201-bfa1-4f05-86f8-3773f8214dbb admin admin] PATH_INFO: `/v3/projects/525b02202b5d4a6286e8ae74eb238500/users/3388bf9a9af74a3289c70ecd11ca52e5/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.147530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4b201-bfa1-4f05-86f8-3773f8214dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=525b02202b5d4a6286e8ae74eb238500, user_id=3388bf9a9af74a3289c70ecd11ca52e5, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.150209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4b201-bfa1-4f05-86f8-3773f8214dbb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.197134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1049/2098] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:38 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:38.201951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.212827 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818df00f-d23a-4468-9f2e-16af728056bf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.217734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.219177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.222053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.222053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.222053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.222330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.229571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1050/2099] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:38 2022] PUT /identity/v3/projects/525b02202b5d4a6286e8ae74eb238500/users/3388bf9a9af74a3289c70ecd11ca52e5/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:09:38.237038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1050/2100] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:38.240790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.241125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.241425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.241717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.251619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813a772-f225-453a-8801-12e6a3bf6426 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.273130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] MFA Rules not processed for user `3388bf9a9af74a3289c70ecd11ca52e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:38.274419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:38.322583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.324145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.324145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.324473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.325142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.326770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.341603 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1051/2101] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:09:38.343777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.352092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1051/2102] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:38.356459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.362324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b1c3c-5068-41c5-be4b-bce8ec613fd5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.372942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d1317-1d00-4b00-99e5-50bb7a58551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.374130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1317-1d00-4b00-99e5-50bb7a58551b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.374486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1317-1d00-4b00-99e5-50bb7a58551b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.375103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d1317-1d00-4b00-99e5-50bb7a58551b admin admin] PATH_INFO: `/v3/projects/8f238c4eb1cf4ec18380c0ee6fe77c6f/users/d120e9219c5e4076a586e23bb5633977/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.378625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89b1aa-3fe0-49ed-8eb9-51f744a43a94 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.386721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.387456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.387748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.388031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.388737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.391499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.399039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1317-1d00-4b00-99e5-50bb7a58551b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f238c4eb1cf4ec18380c0ee6fe77c6f, user_id=d120e9219c5e4076a586e23bb5633977, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.401936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d1317-1d00-4b00-99e5-50bb7a58551b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.534755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1052/2103] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:38 2022] PUT /identity/v3/projects/8f238c4eb1cf4ec18380c0ee6fe77c6f/users/d120e9219c5e4076a586e23bb5633977/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:38.546651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.546950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.547070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.579434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] MFA Rules not processed for user `d120e9219c5e4076a586e23bb5633977`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:38.580516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:38.593469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:38.613155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1052/2104] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:38 2022] POST /identity/v3/projects => generated 375 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:38.616635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.668450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1053/2105] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:38.677833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.679571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:38.723999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ee790-f96b-4f3a-96cb-a017f5fe6014 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.730635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.731642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.732230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.732583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.733390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.736523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.763100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd7dd2-6c48-43ee-9c7d-486eb4014118 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.825692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.826367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.826652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.826929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.848718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1053/2106] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:38 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:38.850321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.867152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c8c6e7-5750-4295-9ba9-2829895e1dbd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.877014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.877967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.878401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.879749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.880979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.882790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.897054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1054/2107] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:38.901375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:38.913214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf19f3f-9ade-4bb6-9fd2-51a2db3600e1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:38.920687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:38.923532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:38.923826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:38.924117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] PATH_INFO: `/v3/projects/5f0009fda21349dc8c062a59f484ab1f/users/ac2d5800a3824180859a129171fcbd53/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:38.938648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.940524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.944485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0009fda21349dc8c062a59f484ab1f, user_id=ac2d5800a3824180859a129171fcbd53, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:38.948347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:38.953389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1054/2108] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:09:39.050752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1055/2109] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:38 2022] PUT /identity/v3/projects/5f0009fda21349dc8c062a59f484ab1f/users/ac2d5800a3824180859a129171fcbd53/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:09:39.054718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:39.068274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4a2fb-4150-4c70-bf65-1019df18a52d service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:39.158924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:39.177266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d642-bd1f-469c-a953-1f9d9edc99ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:39.178484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d642-bd1f-469c-a953-1f9d9edc99ed admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:39.178843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d642-bd1f-469c-a953-1f9d9edc99ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:39.179278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d642-bd1f-469c-a953-1f9d9edc99ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:39.180379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d642-bd1f-469c-a953-1f9d9edc99ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:39.182475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d642-bd1f-469c-a953-1f9d9edc99ed admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:39.195406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1055/2110] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:39.199562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:39.210835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ec680-62e4-4384-b464-7b4b6275d58d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:39.216855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:39.219790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:39.219790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:39.219790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] PATH_INFO: `/v3/projects/5f0009fda21349dc8c062a59f484ab1f/users/ac2d5800a3824180859a129171fcbd53/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:39.221747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0009fda21349dc8c062a59f484ab1f, user_id=ac2d5800a3824180859a129171fcbd53, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:39.224503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:39.237157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1056/2111] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:39 2022] PUT /identity/v3/projects/5f0009fda21349dc8c062a59f484ab1f/users/ac2d5800a3824180859a129171fcbd53/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:39.249096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b94c8-ee4f-4dfd-9352-c6d42e0353b9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:39.255132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b94c8-ee4f-4dfd-9352-c6d42e0353b9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:39.255132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b94c8-ee4f-4dfd-9352-c6d42e0353b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:39.285555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ca6b94c8-ee4f-4dfd-9352-c6d42e0353b9 None None] MFA Rules not processed for user `ac2d5800a3824180859a129171fcbd53`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:39.287615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca6b94c8-ee4f-4dfd-9352-c6d42e0353b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:39.367916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1056/2112] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:39.379654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:39.444177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4b0e9-edc4-432e-9b0c-035b8236d156 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:39.448025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:39.448697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:39.448981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:39.449258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:39.451310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:39.489921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:39.520213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:39.532858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:39.532858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1057/2113] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:40.010262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:40.013562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:40.013562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:40.031251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] MFA Rules not processed for user `2311f0b1e7074500a6964504a7aee218`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:40.036998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:40.099159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1057/2114] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:41.046658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:41.876295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:42.776176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:42.778001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:42.791080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f6f7d-2574-4123-993f-9f0f1e44d6b5 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:42.794934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:42.795693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:42.796042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:42.796381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:42.875636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:42.877126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:42.887267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1058/2115] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:09:44.502072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:44.511718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43d31-8284-4c09-aec5-338ce2330589 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:44.515298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd913f-eb10-4c61-ab93-31d288143c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:44.515970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd913f-eb10-4c61-ab93-31d288143c8e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:44.516254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd913f-eb10-4c61-ab93-31d288143c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:44.516528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd913f-eb10-4c61-ab93-31d288143c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:44.525935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd913f-eb10-4c61-ab93-31d288143c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:44.527599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd913f-eb10-4c61-ab93-31d288143c8e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:44.538999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1058/2116] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:09:50.234380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:50.334224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1096658-225c-4a63-b67b-2619ac2e34a1 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:50.339960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:50.342302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:50.346332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:50.346629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] PATH_INFO: `/v3/users/cbd10fb86c464c4c8265726d3f27203c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:50.348968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd10fb86c464c4c8265726d3f27203c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:50.352395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:50.877694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:50.886949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd10fb86c464c4c8265726d3f27203c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:50.887815 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd10fb86c464c4c8265726d3f27203c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:50.958325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd10fb86c464c4c8265726d3f27203c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:50.965920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd10fb86c464c4c8265726d3f27203c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:50.979135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd10fb86c464c4c8265726d3f27203c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:50.985693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1059/2117] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:50 2022] DELETE /identity/v3/users/cbd10fb86c464c4c8265726d3f27203c => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:50.990533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:50.991094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:50.991094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:51.009507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:51.027092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:51.074463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1059/2118] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:51.085382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:51.086987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:51.148463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfdc44-6408-4833-a5f3-2705c40988c7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:51.152491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:51.153160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:51.153449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:51.153729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:51.166063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:51.168421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:51.183288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1060/2119] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:09:51.691770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:51.693293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:51.706464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33f8a6-d3af-42a7-a089-42d41612a0e3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:51.712372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:51.713241 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:51.713520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:51.713887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] PATH_INFO: `/v3/projects/495f0db42be04e6dbf79141ab7fe8a1c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:51.715719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=495f0db42be04e6dbf79141ab7fe8a1c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:51.720515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:51.875568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495f0db42be04e6dbf79141ab7fe8a1c', 'request_id': 'req-21c229fe-bd88-44da-8bab-e577bd8ffdf3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:51.935327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495f0db42be04e6dbf79141ab7fe8a1c', 'request_id': 'req-21c229fe-bd88-44da-8bab-e577bd8ffdf3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:51.942154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495f0db42be04e6dbf79141ab7fe8a1c', 'request_id': 'req-21c229fe-bd88-44da-8bab-e577bd8ffdf3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:51.953596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] The token cache is being invalidate because project 495f0db42be04e6dbf79141ab7fe8a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:51.953883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:51.954532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:51.956125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1060/2120] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:51 2022] DELETE /identity/v3/projects/495f0db42be04e6dbf79141ab7fe8a1c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:51.960852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:51.970801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956986fd-6a59-4a77-a83a-523384a3302f service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:52.032055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:52.033195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:52.033480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:52.033762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] PATH_INFO: `/v3/users/3388bf9a9af74a3289c70ecd11ca52e5` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:52.034955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3388bf9a9af74a3289c70ecd11ca52e5)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:52.037133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:52.213970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3388bf9a9af74a3289c70ecd11ca52e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:52.214735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3388bf9a9af74a3289c70ecd11ca52e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:52.279132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3388bf9a9af74a3289c70ecd11ca52e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:52.281723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:52.287732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3388bf9a9af74a3289c70ecd11ca52e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:52.299923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3388bf9a9af74a3289c70ecd11ca52e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:52.307110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1061/2121] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:51 2022] DELETE /identity/v3/users/3388bf9a9af74a3289c70ecd11ca52e5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:52.800632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:52.809413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:52.821792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c229fe-bd88-44da-8bab-e577bd8ffdf3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:52.888214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:52.894116 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:52.897873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:52.897873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:52.897873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] PATH_INFO: `/v3/projects/525b02202b5d4a6286e8ae74eb238500` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:52.897873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=525b02202b5d4a6286e8ae74eb238500)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:52.899280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:53.100985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525b02202b5d4a6286e8ae74eb238500', 'request_id': 'req-1680adde-2c3f-4027-b767-8d3797564180'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.221716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525b02202b5d4a6286e8ae74eb238500', 'request_id': 'req-1680adde-2c3f-4027-b767-8d3797564180'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.230914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525b02202b5d4a6286e8ae74eb238500', 'request_id': 'req-1680adde-2c3f-4027-b767-8d3797564180'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.247581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] The token cache is being invalidate because project 525b02202b5d4a6286e8ae74eb238500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:53.247878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.248588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.250450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1061/2122] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:52 2022] DELETE /identity/v3/projects/525b02202b5d4a6286e8ae74eb238500 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:53.254744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:53.258519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:53.273172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4fc95-9c26-4871-81e3-9bb29188479e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:53.338516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:53.339327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:53.339639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:53.339916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] PATH_INFO: `/v3/users/ac2d5800a3824180859a129171fcbd53` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:53.341101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2d5800a3824180859a129171fcbd53)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:53.343007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:53.527130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2d5800a3824180859a129171fcbd53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.527814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2d5800a3824180859a129171fcbd53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.570092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2d5800a3824180859a129171fcbd53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.578665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2d5800a3824180859a129171fcbd53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.593366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2d5800a3824180859a129171fcbd53'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:53.600213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1062/2123] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:53 2022] DELETE /identity/v3/users/ac2d5800a3824180859a129171fcbd53 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:09:54.051854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.053956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.067284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680adde-2c3f-4027-b767-8d3797564180 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.135590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.136344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.136633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.136917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] PATH_INFO: `/v3/projects/5f0009fda21349dc8c062a59f484ab1f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.138154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0009fda21349dc8c062a59f484ab1f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.140156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.198767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.200260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.213827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee7798-0897-4faa-98cd-88943b78be6e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.218592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.219505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.219794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.220073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] PATH_INFO: `/v3/users/d120e9219c5e4076a586e23bb5633977` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.221271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d120e9219c5e4076a586e23bb5633977)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.223260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.258602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0009fda21349dc8c062a59f484ab1f', 'request_id': 'req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.341329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0009fda21349dc8c062a59f484ab1f', 'request_id': 'req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.348249 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0009fda21349dc8c062a59f484ab1f', 'request_id': 'req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.360350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] The token cache is being invalidate because project 5f0009fda21349dc8c062a59f484ab1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:54.360639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.361287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7db0a28-deed-4599-a76b-e0cac9ad1d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.362898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1062/2124] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:54 2022] DELETE /identity/v3/projects/5f0009fda21349dc8c062a59f484ab1f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:54.368267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be8967-e382-45f5-9b7b-df54c7b0cf14 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.369564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be8967-e382-45f5-9b7b-df54c7b0cf14 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.369564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be8967-e382-45f5-9b7b-df54c7b0cf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.389982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-15be8967-e382-45f5-9b7b-df54c7b0cf14 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:54.391167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-15be8967-e382-45f5-9b7b-df54c7b0cf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.401372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd120e9219c5e4076a586e23bb5633977'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.405254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd120e9219c5e4076a586e23bb5633977'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.405254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:54.405882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:54.457267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1063/2125] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:54.466405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.466694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.466974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.471748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd120e9219c5e4076a586e23bb5633977'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.479947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd120e9219c5e4076a586e23bb5633977'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.485122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:54.486338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.493136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48893547-f012-4109-aa97-96aad0d9b839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd120e9219c5e4076a586e23bb5633977'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:54.495238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1064/2126] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:54.499103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.499595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1063/2127] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:09:54 2022] DELETE /identity/v3/users/d120e9219c5e4076a586e23bb5633977 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:09:54.503631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.503923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.504204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.511003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385336d-2cc3-4692-baaa-756353fc4914 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.523561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:54.524778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.568925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.569590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.569880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.570157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.570882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:09:54.571277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.572931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.587671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1065/2128] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:09:54.587951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1064/2129] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:54.593771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.595468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.597289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.598671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:54.609079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31765953-6985-4cc9-a45a-766e8ed332aa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.614896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.616346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.618451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.618451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.618451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.619746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.675865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de034c-65a2-4a53-990c-657d1a1fd7be None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.680824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.681477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.681757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.682034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.691098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.692585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.704340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1065/2130] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:09:54.733296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1066/2131] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:54 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:54.737819 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.747978 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0eb36e-3167-41dd-baec-e567598849d7 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.808080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.808879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.809168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.809446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.810148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.812086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.895663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:54.903455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1066/2132] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:54 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:54.908655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.920689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f53f0-f3a1-4d24-bb36-e274e387b1fe admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.925813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.926867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.926867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.927051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.928808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.929574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.940839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1067/2133] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:54.945624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.955866 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedc81e-bcdd-46d7-8989-eb0c8029acf4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.962258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.962938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.963223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.963542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:54.964273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:54.965865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:54.977107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1067/2134] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:09:54.981341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:54.991613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553896ea-6d14-4cbc-8972-1532a26d93e4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:54.996407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2904-5e35-4ce3-9773-6221982870ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:54.997346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2904-5e35-4ce3-9773-6221982870ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:54.997633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2904-5e35-4ce3-9773-6221982870ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:54.997951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2904-5e35-4ce3-9773-6221982870ae admin admin] PATH_INFO: `/v3/projects/83edf57b085d4c2ebf2994e8685faed3/users/81d9ecab365d43ebb71da4661ddc4034/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.014875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2904-5e35-4ce3-9773-6221982870ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=83edf57b085d4c2ebf2994e8685faed3, user_id=81d9ecab365d43ebb71da4661ddc4034, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.019000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2904-5e35-4ce3-9773-6221982870ae admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:55.131560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1068/2135] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:54 2022] PUT /identity/v3/projects/83edf57b085d4c2ebf2994e8685faed3/users/81d9ecab365d43ebb71da4661ddc4034/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:55.135311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:55.135311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:55.147883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.148180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.148465 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.158861 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451ee09-39e4-493c-a9e3-8989cd9b616c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:55.182346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] MFA Rules not processed for user `81d9ecab365d43ebb71da4661ddc4034`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:55.183621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:55.219025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:55.219920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.220206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.220480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] PATH_INFO: `/v3/projects/8f238c4eb1cf4ec18380c0ee6fe77c6f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.221654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f238c4eb1cf4ec18380c0ee6fe77c6f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.222709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:55.223575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:55.267767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1069/2136] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:55.280345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:55.281804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:55.360490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f238c4eb1cf4ec18380c0ee6fe77c6f', 'request_id': 'req-80cacaa6-c101-4071-828a-8d6cb71e3be0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:55.363441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29721b-f5aa-443a-adc4-877486167720 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:55.367445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f238c4eb1cf4ec18380c0ee6fe77c6f', 'request_id': 'req-80cacaa6-c101-4071-828a-8d6cb71e3be0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:55.374693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f238c4eb1cf4ec18380c0ee6fe77c6f', 'request_id': 'req-80cacaa6-c101-4071-828a-8d6cb71e3be0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:55.415678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e41263-fd45-4361-9b5b-2f0cf4d6825a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:55.416338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e41263-fd45-4361-9b5b-2f0cf4d6825a service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.416665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e41263-fd45-4361-9b5b-2f0cf4d6825a service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.416906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e41263-fd45-4361-9b5b-2f0cf4d6825a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.473261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] The token cache is being invalidate because project 8f238c4eb1cf4ec18380c0ee6fe77c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:09:55.473831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e41263-fd45-4361-9b5b-2f0cf4d6825a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.473948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:55.474222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-80cacaa6-c101-4071-828a-8d6cb71e3be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:09:55.475432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e41263-fd45-4361-9b5b-2f0cf4d6825a service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:55.476515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1068/2137] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:09:55 2022] DELETE /identity/v3/projects/8f238c4eb1cf4ec18380c0ee6fe77c6f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:55.481818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.482106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.482385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.484800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1070/2138] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:09:55.501494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:55.502853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:55.577410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1069/2139] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:55.580445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.580804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.581154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.600819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:55.602092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:55.610398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1071/2140] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:09:55.614712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:55.626005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a9230-fb00-40a5-b998-09c63b3335b0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:55.630664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:55.631408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.631825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.632104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.632855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:09:55.633161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.634739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:55.647958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1070/2141] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:09:55.654203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:55.664951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3dd36b-1726-45e8-bc0f-9208c62d019e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:55.670238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:55.670786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.671101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.671386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.673645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.675867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:55.761314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:55.768653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1072/2142] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:09:55 2022] POST /identity/v3/projects => generated 399 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:09:55.773076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:55.779665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:09:55.783926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e1a9-862f-4e4c-9eaf-6138e08c295d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:55.841796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:55.842468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.842750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.843024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.843765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.845638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:55.969438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1071/2143] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:09:55 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:09:55.973668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:55.984702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87da9176-9918-4072-bc5f-96939e0ee635 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:55.989278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:55.989913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:55.990319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:55.991108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:55.991877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:55.993449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:56.004093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1073/2144] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:56.009425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:56.019600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dee998-da34-4795-8273-6ed0707a4de7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:56.025111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8539a3-bdc3-42fe-9a1d-7d3d27a39967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:56.025781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8539a3-bdc3-42fe-9a1d-7d3d27a39967 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:56.026069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8539a3-bdc3-42fe-9a1d-7d3d27a39967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:56.026346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8539a3-bdc3-42fe-9a1d-7d3d27a39967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:56.027072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8539a3-bdc3-42fe-9a1d-7d3d27a39967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:56.028695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8539a3-bdc3-42fe-9a1d-7d3d27a39967 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:56.039991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1072/2145] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:09:56.044195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:56.055433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288e7fa-5778-4ca8-8b40-9711cf30286d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:56.060172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:56.061053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:56.061340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:56.061615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] PATH_INFO: `/v3/projects/600f137456594e8488a422c9eb98f9f5/users/e56aee42457d48d6ab8540c5346ca4e7/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:56.078911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600f137456594e8488a422c9eb98f9f5, user_id=e56aee42457d48d6ab8540c5346ca4e7, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:56.081690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:56.175075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1074/2146] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:09:56 2022] PUT /identity/v3/projects/600f137456594e8488a422c9eb98f9f5/users/e56aee42457d48d6ab8540c5346ca4e7/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:09:56.185863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:56.186158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:56.186462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:56.214705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] MFA Rules not processed for user `e56aee42457d48d6ab8540c5346ca4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:09:56.215974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:56.282773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1073/2147] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:09:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:09:56.294361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:09:56.296093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:09:56.358944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812da40-b05d-429f-862f-0a22a65e3444 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:09:56.362641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d4aa3-30a8-4b41-bf74-4653e3e985ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:09:56.363299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4aa3-30a8-4b41-bf74-4653e3e985ff service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:09:56.363604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4aa3-30a8-4b41-bf74-4653e3e985ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:09:56.363874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4aa3-30a8-4b41-bf74-4653e3e985ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:09:56.373860 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4aa3-30a8-4b41-bf74-4653e3e985ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:09:56.375302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4aa3-30a8-4b41-bf74-4653e3e985ff service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:09:56.385295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1075/2148] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:09:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:10:06.279708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:06.380263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:11.847925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:11.914845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cd7e9-8efb-4535-87f3-3dc25ca6f508 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:11.920069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:11.920953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:11.921240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:11.921523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] PATH_INFO: `/v3/users/e56aee42457d48d6ab8540c5346ca4e7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:11.922751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56aee42457d48d6ab8540c5346ca4e7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:11.924726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:12.175457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56aee42457d48d6ab8540c5346ca4e7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:12.176131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56aee42457d48d6ab8540c5346ca4e7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:12.265390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56aee42457d48d6ab8540c5346ca4e7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:12.273159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56aee42457d48d6ab8540c5346ca4e7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:12.285466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56aee42457d48d6ab8540c5346ca4e7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:12.291956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1074/2149] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:10:11 2022] DELETE /identity/v3/users/e56aee42457d48d6ab8540c5346ca4e7 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:10:12.299475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:12.299965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:12.300418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:12.323835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:12.323835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:12.382798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1076/2150] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:10:12.392657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:12.398167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:12.459911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ae1d7-f933-4a97-bb33-c10582591c39 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:12.463548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:12.464733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:12.465159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:12.465442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:12.474492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:12.475969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:12.486157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1075/2151] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:10:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:10:12.940985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:12.942877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:12.957260 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7b218-c3c2-445e-8143-5449ecc6b555 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:12.962197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:12.963079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:12.963389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:12.963682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] PATH_INFO: `/v3/projects/600f137456594e8488a422c9eb98f9f5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:12.964983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600f137456594e8488a422c9eb98f9f5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:12.967564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:13.152134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600f137456594e8488a422c9eb98f9f5', 'request_id': 'req-1c43c4d4-92c1-402b-87d7-d00947ff3c85'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:13.236135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600f137456594e8488a422c9eb98f9f5', 'request_id': 'req-1c43c4d4-92c1-402b-87d7-d00947ff3c85'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:13.256235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600f137456594e8488a422c9eb98f9f5', 'request_id': 'req-1c43c4d4-92c1-402b-87d7-d00947ff3c85'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:13.273250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] The token cache is being invalidate because project 600f137456594e8488a422c9eb98f9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:10:13.274719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:13.274719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c43c4d4-92c1-402b-87d7-d00947ff3c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:13.275857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1077/2152] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:10:12 2022] DELETE /identity/v3/projects/600f137456594e8488a422c9eb98f9f5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:10:17.732543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:17.733996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:17.761545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:17.805107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f0b13-b5f6-494c-a853-b756e3a06a21 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:17.870663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:17.871303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:17.871632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:17.871910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:17.873771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:17.885734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:17.887270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:17.898038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1076/2153] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:10:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:10:22.342071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab9c96-b09c-4e3a-b78c-050e581eab1f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:22.342608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab9c96-b09c-4e3a-b78c-050e581eab1f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:22.342735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab9c96-b09c-4e3a-b78c-050e581eab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:22.361834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-46ab9c96-b09c-4e3a-b78c-050e581eab1f None None] MFA Rules not processed for user `8498989e673d445dbc70aa9df55f391b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:22.363314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-46ab9c96-b09c-4e3a-b78c-050e581eab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:22.421466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1078/2154] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:10:27.915863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=48 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:32.414935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:46.890069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:46.891748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:47.000811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170be156-08d2-4cb3-8ed0-3facabd6cd63 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:47.006671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:47.007636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:47.007932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:47.008188 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] PATH_INFO: `/v3/users/81d9ecab365d43ebb71da4661ddc4034` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:47.009448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d9ecab365d43ebb71da4661ddc4034)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:47.011445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:47.293713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d9ecab365d43ebb71da4661ddc4034'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:47.294437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d9ecab365d43ebb71da4661ddc4034'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:47.424656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d9ecab365d43ebb71da4661ddc4034'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:47.435903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d9ecab365d43ebb71da4661ddc4034'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:47.450230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d9ecab365d43ebb71da4661ddc4034'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:47.456870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1077/2155] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:10:46 2022] DELETE /identity/v3/users/81d9ecab365d43ebb71da4661ddc4034 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:10:47.465471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:47.465471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:47.465471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:47.485435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:47.487498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:47.574163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1079/2156] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:10:47.584003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:47.586189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:47.671463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d965-dac8-45da-aacc-ee0c749075ed admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:47.677868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4eb69-5b75-4b39-8a86-17adbf4550fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:47.680333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4eb69-5b75-4b39-8a86-17adbf4550fe service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:47.680522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4eb69-5b75-4b39-8a86-17adbf4550fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:47.680918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4eb69-5b75-4b39-8a86-17adbf4550fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:47.698284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4eb69-5b75-4b39-8a86-17adbf4550fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:47.702629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4eb69-5b75-4b39-8a86-17adbf4550fe service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:47.719754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1078/2157] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:10:48.802339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:48.805083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:48.816918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237ac7-3b67-4787-993a-7285ded777a7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:48.823133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:48.824263 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:48.824631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:48.824982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] PATH_INFO: `/v3/projects/83edf57b085d4c2ebf2994e8685faed3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:48.826520 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83edf57b085d4c2ebf2994e8685faed3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:48.828554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:49.061756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83edf57b085d4c2ebf2994e8685faed3', 'request_id': 'req-80ca1e56-b236-4c5e-9416-43eefde090d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:49.201604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83edf57b085d4c2ebf2994e8685faed3', 'request_id': 'req-80ca1e56-b236-4c5e-9416-43eefde090d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:49.208730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83edf57b085d4c2ebf2994e8685faed3', 'request_id': 'req-80ca1e56-b236-4c5e-9416-43eefde090d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:49.220969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] The token cache is being invalidate because project 83edf57b085d4c2ebf2994e8685faed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:10:49.221256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:49.221945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-80ca1e56-b236-4c5e-9416-43eefde090d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:10:49.223634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1080/2158] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:10:48 2022] DELETE /identity/v3/projects/83edf57b085d4c2ebf2994e8685faed3 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:10:49.229229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:49.229744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:49.229840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:49.248847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:49.250070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:49.277466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:49.345668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1079/2159] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:10:49.353608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:49.354186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:49.354652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:49.377329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:49.378680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:49.393411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1081/2160] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:10:49.400779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:49.416598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256bf32-dbe1-4485-82ed-8fe23d98e13f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:49.422288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:49.423008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:49.423322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:49.423712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:49.424454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:10:49.424774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:49.426388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:49.442287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1080/2161] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:10:49.447590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:49.459317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bec95-8276-42c0-8a4d-00249fb48219 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:49.464639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:49.465776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:49.465905 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:49.481486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:49.481486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:49.481486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:49.649930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1082/2162] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:10:49 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:10:49.652133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:49.663240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5939fe8-358e-4175-94c0-9f9ac1c5cfd7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:49.773612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:49.774354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:49.775717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:49.776107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:49.777692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:49.780372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:49.934510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:49.935138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1081/2163] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:10:49 2022] POST /identity/v3/users => generated 288 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:10:49.951192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098196e-0b68-4f27-8825-d3a63d3e512a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:49.963541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:49.965033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:49.965642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:49.965929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:49.966914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:49.968713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:49.983415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1083/2164] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:10:49.988060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:50.007993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429404b-ef7b-44f8-8c1b-4ff3c81dec63 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:50.027791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d5a25-2be3-4626-a5eb-75cd8d7a5a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:50.027791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d5a25-2be3-4626-a5eb-75cd8d7a5a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:50.027791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d5a25-2be3-4626-a5eb-75cd8d7a5a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:50.027791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d5a25-2be3-4626-a5eb-75cd8d7a5a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:50.028386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d5a25-2be3-4626-a5eb-75cd8d7a5a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:50.028386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d5a25-2be3-4626-a5eb-75cd8d7a5a4d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:50.036830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1082/2165] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:10:50.045175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:50.060221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2b4dd-fd49-46a6-8cf0-312465d5d5dd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:50.066678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:50.067312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:50.067689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:50.075254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] PATH_INFO: `/v3/projects/42a4b66cbb644093b01587e481741a86/users/1ff319ceadf84863850254e50774ba5b/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:50.089436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a4b66cbb644093b01587e481741a86, user_id=1ff319ceadf84863850254e50774ba5b, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:50.092388 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:50.279583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1084/2166] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:10:50 2022] PUT /identity/v3/projects/42a4b66cbb644093b01587e481741a86/users/1ff319ceadf84863850254e50774ba5b/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:10:50.285066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b65d2-bd72-4275-8443-0aca257b9a15 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:50.285602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b65d2-bd72-4275-8443-0aca257b9a15 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:50.285773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b65d2-bd72-4275-8443-0aca257b9a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:50.328864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0d9b65d2-bd72-4275-8443-0aca257b9a15 None None] MFA Rules not processed for user `1ff319ceadf84863850254e50774ba5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:50.330560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0d9b65d2-bd72-4275-8443-0aca257b9a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:50.461814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1083/2167] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:10:50.475313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:50.478403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:50.517566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:50.572344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8feee7-267e-442c-9370-af6ccbc28079 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:50.580214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:50.581042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:50.581408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:50.581882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:50.596511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:50.598058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:50.611779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1085/2168] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:10:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:10:50.893090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:50.893441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:50.893800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:50.914055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:50.915311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:50.955733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:10:51.003175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1084/2169] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:10:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:10:51.289977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:51.295333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:51.378216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a02d6-2f62-48f9-bd47-33e1d2547a0f service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:51.384418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384d0fa-582d-44ba-9d61-5c35d88dfcdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:51.385473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384d0fa-582d-44ba-9d61-5c35d88dfcdc service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:51.385473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384d0fa-582d-44ba-9d61-5c35d88dfcdc service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:51.387883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384d0fa-582d-44ba-9d61-5c35d88dfcdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:51.394859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384d0fa-582d-44ba-9d61-5c35d88dfcdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:51.396477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384d0fa-582d-44ba-9d61-5c35d88dfcdc service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:51.404966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1086/2170] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:10:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:10:51.476183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:10:51.478640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:51.579862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744858c1-e573-49a2-93e8-635794ff20bc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:10:51.587007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:10:51.592135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:51.592135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:51.592135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:51.592135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:51.724660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:10:51.726919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd0143-59ea-468e-8946-b36db8173a85 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:10:51.739008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1085/2171] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:10:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:10:53.920158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d49e0c-a116-40dd-9c93-23b97275f4e6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:10:53.920991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d49e0c-a116-40dd-9c93-23b97275f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:10:53.920991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d49e0c-a116-40dd-9c93-23b97275f4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:10:53.950799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-44d49e0c-a116-40dd-9c93-23b97275f4e6 None None] MFA Rules not processed for user `1ff319ceadf84863850254e50774ba5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:10:53.952153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-44d49e0c-a116-40dd-9c93-23b97275f4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:10:53.962294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1087/2172] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:10:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:01.792125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=67 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:03.939549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=52 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:13.889381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ff80f-5a80-426c-a380-69e85728716c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:13.894447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ff80f-5a80-426c-a380-69e85728716c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:13.894447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ff80f-5a80-426c-a380-69e85728716c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:13.894447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1086/2173] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:11:13.899977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6d855-2d50-4c47-8234-5d1e808788f3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:13.900470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6d855-2d50-4c47-8234-5d1e808788f3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:13.900689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6d855-2d50-4c47-8234-5d1e808788f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:13.946128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-81d6d855-2d50-4c47-8234-5d1e808788f3 None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:13.947463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-81d6d855-2d50-4c47-8234-5d1e808788f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:14.033486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1088/2174] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:11:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:11:14.046969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ff80f-5a80-426c-a380-69e85728716c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:14.050407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9e0ff80f-5a80-426c-a380-69e85728716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:14.122212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ff80f-5a80-426c-a380-69e85728716c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:14.127727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cb98f-5484-43a2-bd09-bb33a225e9de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:14.128807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb98f-5484-43a2-bd09-bb33a225e9de service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.129038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb98f-5484-43a2-bd09-bb33a225e9de service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.129412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cb98f-5484-43a2-bd09-bb33a225e9de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.142343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cb98f-5484-43a2-bd09-bb33a225e9de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:14.144292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cb98f-5484-43a2-bd09-bb33a225e9de service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:14.156923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1087/2175] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:14.567741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b369-4bed-49f8-9634-a84a7f7acb80 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.568028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b369-4bed-49f8-9634-a84a7f7acb80 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.568306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98b369-4bed-49f8-9634-a84a7f7acb80 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.569696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1089/2176] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:11:14.574456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ddf28-85c7-45a6-9605-dda248c335e5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.574736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ddf28-85c7-45a6-9605-dda248c335e5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.575009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ddf28-85c7-45a6-9605-dda248c335e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.593785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ef9ddf28-85c7-45a6-9605-dda248c335e5 None None] MFA Rules not processed for user `6aae965af8284363820254c9286adf96`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:14.595011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ef9ddf28-85c7-45a6-9605-dda248c335e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:14.602076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1088/2177] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:14.615178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b369-4bed-49f8-9634-a84a7f7acb80 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:14.628486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98b369-4bed-49f8-9634-a84a7f7acb80 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:14.632139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07da751-b819-45d7-b464-cdf3446e0e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:14.632795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da751-b819-45d7-b464-cdf3446e0e53 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.633083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da751-b819-45d7-b464-cdf3446e0e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.633363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da751-b819-45d7-b464-cdf3446e0e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.641458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07da751-b819-45d7-b464-cdf3446e0e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:14.643273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07da751-b819-45d7-b464-cdf3446e0e53 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:14.652369 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1090/2178] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:14.744304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04c24-9bc7-4420-8e36-eab25da44b86 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.744618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04c24-9bc7-4420-8e36-eab25da44b86 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.744900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba04c24-9bc7-4420-8e36-eab25da44b86 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.747454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1089/2179] 10.222.0.58 () {54 vars in 921 bytes} [Thu Jun 16 18:11:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:11:14.754236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9eec8-0507-4291-b80f-a773cb8910d1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.754550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9eec8-0507-4291-b80f-a773cb8910d1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.754834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9eec8-0507-4291-b80f-a773cb8910d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.775182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d1d9eec8-0507-4291-b80f-a773cb8910d1 None None] MFA Rules not processed for user `9279195b1ed64901a63a3fce8d0bb610`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:14.777051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d1d9eec8-0507-4291-b80f-a773cb8910d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:14.786924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1091/2180] 10.222.0.58 () {58 vars in 1018 bytes} [Thu Jun 16 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:14.798798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba04c24-9bc7-4420-8e36-eab25da44b86 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:14.813727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba04c24-9bc7-4420-8e36-eab25da44b86 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:14.821266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:14.822199 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:14.822717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:14.823009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:14.837423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:14.838895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:14.852857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1090/2181] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:19.782998 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c781c-d450-4389-91d3-873f38c62b7a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:19.783514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c781c-d450-4389-91d3-873f38c62b7a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:19.788413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c781c-d450-4389-91d3-873f38c62b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:19.803207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5b1c781c-d450-4389-91d3-873f38c62b7a None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:19.804498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5b1c781c-d450-4389-91d3-873f38c62b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:19.813305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1092/2182] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:11:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:19.835570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:19.839196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f09f0e-b381-4830-aba4-e07825157c9e service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:19.852877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:19.853599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:19.853881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:19.854109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:19.865904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:19.867726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:19.879242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1091/2183] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:11:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:11:20.988294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae00f4e-a761-464b-b08f-01d591891c95 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:20.988774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae00f4e-a761-464b-b08f-01d591891c95 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:20.988879 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae00f4e-a761-464b-b08f-01d591891c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:21.019078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fae00f4e-a761-464b-b08f-01d591891c95 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:21.020632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fae00f4e-a761-464b-b08f-01d591891c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:21.031514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1093/2184] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:11:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:21.046643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:21.066689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90804b65-99bc-4cdb-8873-bba4b2c815bd service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:21.081212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:21.082158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:21.082685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:21.083942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:21.097586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:21.099955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:21.116895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1092/2185] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:11:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:11:21.519963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:21.520857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:21.521486 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:21.561492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:21.563765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:21.587526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1094/2186] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jun 16 18:11:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:21.604618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:21.617557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79d9b2-9f96-4511-b554-99cc7d54a883 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:21.624908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8c40d-a0b6-4c9d-ad1b-7ecce33469c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:21.625878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c40d-a0b6-4c9d-ad1b-7ecce33469c2 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:21.626348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c40d-a0b6-4c9d-ad1b-7ecce33469c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:21.627030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8c40d-a0b6-4c9d-ad1b-7ecce33469c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:21.638014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c40d-a0b6-4c9d-ad1b-7ecce33469c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:21.641337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8c40d-a0b6-4c9d-ad1b-7ecce33469c2 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:21.654360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1093/2187] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:11:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:11:22.354406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:22.357161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:22.389508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e9195-e11c-4ff9-8a4e-5ebd2f5396e2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:22.398238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:22.399926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:22.400618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:22.401302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:22.410048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:22.435628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:22.437082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:22.450746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1095/2188] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:23.140489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:23.148518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:23.148518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:23.162243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:23.163604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:23.172323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1094/2189] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:11:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:23.185169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:23.197651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e375a-7fe6-4512-a7d1-c5093af6a015 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:23.203219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd275cb-47b2-40a7-94d9-85b5d1a34e99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:23.203936 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd275cb-47b2-40a7-94d9-85b5d1a34e99 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:23.204229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd275cb-47b2-40a7-94d9-85b5d1a34e99 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:23.204516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd275cb-47b2-40a7-94d9-85b5d1a34e99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:23.214630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd275cb-47b2-40a7-94d9-85b5d1a34e99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:23.216835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd275cb-47b2-40a7-94d9-85b5d1a34e99 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:23.227638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1096/2190] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:11:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:11:23.706041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:23.722988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbc228-de28-4d9c-b46b-a2e298b37a7d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:23.727271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:23.727989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:23.728281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:23.728595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:23.731153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:23.747705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:23.747705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:23.764615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1095/2191] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:25.010575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:25.011343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:25.011343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:25.041602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:25.045432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:25.060136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1097/2192] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jun 16 18:11:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:25.088644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:25.101946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a222b3-1da7-4824-90a7-30eb8aa07f54 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:25.110113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745033e-8b8d-4d37-83f4-2d409859f9c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:25.110113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745033e-8b8d-4d37-83f4-2d409859f9c6 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:25.111213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745033e-8b8d-4d37-83f4-2d409859f9c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:25.111213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745033e-8b8d-4d37-83f4-2d409859f9c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:25.120809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745033e-8b8d-4d37-83f4-2d409859f9c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:25.123750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745033e-8b8d-4d37-83f4-2d409859f9c6 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:25.134693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1096/2193] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:11:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:11:30.819958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:30.894079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f041b-a9d8-4c8b-a22f-eb8c7d8b9c84 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:30.898834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:30.900188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:30.900188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:30.900511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] PATH_INFO: `/v3/users/1ff319ceadf84863850254e50774ba5b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:30.901532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff319ceadf84863850254e50774ba5b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:30.904518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:30.911293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:31.335886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff319ceadf84863850254e50774ba5b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:31.354244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff319ceadf84863850254e50774ba5b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:31.362608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff319ceadf84863850254e50774ba5b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:31.641280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff319ceadf84863850254e50774ba5b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:31.641976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff319ceadf84863850254e50774ba5b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:31.649188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1098/2194] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:11:30 2022] DELETE /identity/v3/users/1ff319ceadf84863850254e50774ba5b => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:11:31.654276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:31.654891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:31.654891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:31.674329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:31.675604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:31.743403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1097/2195] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:31.756115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:31.757707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:31.840348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710d0f3-ea14-4ed1-aa77-169cb8347b8c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:31.844545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:31.845220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:31.847764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:31.847764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:31.860626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:31.862467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:31.881056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1099/2196] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:32.322999 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:32.326038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:32.346762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8614e-86b7-4006-8b99-61fa4aec01b3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:32.353158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:32.354493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:32.354839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:32.355221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] PATH_INFO: `/v3/projects/42a4b66cbb644093b01587e481741a86` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:32.356763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a4b66cbb644093b01587e481741a86)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:32.358969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:32.365846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:32.529234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a4b66cbb644093b01587e481741a86', 'request_id': 'req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:32.535447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a4b66cbb644093b01587e481741a86', 'request_id': 'req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:32.542831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a4b66cbb644093b01587e481741a86', 'request_id': 'req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:32.681409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] The token cache is being invalidate because project 42a4b66cbb644093b01587e481741a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:11:32.682258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:32.682369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:11:32.684088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1098/2197] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:11:32 2022] DELETE /identity/v3/projects/42a4b66cbb644093b01587e481741a86 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:11:34.978619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:34.980387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:35.042681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea21bb-69b4-4bd7-91e1-7d5f23e20528 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:35.086863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:35.095761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:35.096410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:35.096691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:35.096965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:35.098746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:35.161173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:35.162631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe74d65-e309-4859-8682-e98d05d92cfa service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:35.172542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1100/2198] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:42.677931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:44.108646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:44.113165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:44.220914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5a5a6-92a3-4b44-9aa4-b2f180b7e55e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:44.227921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:44.233854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:44.233854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:44.233854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:44.233854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:44.322445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:44.324094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:44.334492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1099/2199] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:45.217321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=28 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:11:48.896956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:48.897653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:48.897653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:48.917352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:11:48.919321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:48.988740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1101/2200] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jun 16 18:11:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:11:49.002853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:49.005190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:49.090428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eeb5a8-37d9-4a79-87dd-e97b4ed97523 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:49.095410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c512d-7bbe-4ee3-9fd9-eb0dafcdeb8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:49.096091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c512d-7bbe-4ee3-9fd9-eb0dafcdeb8b service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:49.096377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c512d-7bbe-4ee3-9fd9-eb0dafcdeb8b service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:49.096664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c512d-7bbe-4ee3-9fd9-eb0dafcdeb8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:49.106571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c512d-7bbe-4ee3-9fd9-eb0dafcdeb8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:49.108814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c512d-7bbe-4ee3-9fd9-eb0dafcdeb8b service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:49.117905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1100/2201] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:11:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:11:57.595954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:11:57.597795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:57.664034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e352ca1-739c-4933-8449-e6f5310bb135 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:11:57.667861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:11:57.668515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:11:57.668799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:11:57.669092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:11:57.671359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:11:57.681855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:11:57.683275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:11:57.692338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1102/2202] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:11:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:11:59.114172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:12:00.881531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:00.882044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:00.882044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:00.900936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] MFA Rules not processed for user `70d6f15465b84182a28d96d1808a9660`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:12:00.902160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:00.909679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1101/2203] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:12:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:12:07.664182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:12:07.666376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:07.690771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c2e40-2221-462b-bce5-238e6b454434 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:12:07.699005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:12:07.700874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:07.701212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:07.701543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:07.704133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:07.719446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:12:07.721518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:12:07.738059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1103/2204] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:12:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:12:10.895921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:12:12.983889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:12:12.998312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f2f5f-67f8-4e24-9594-c6293a7f4c00 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:12:13.004825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:12:13.005510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:13.005798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:13.006093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:13.008197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:13.094356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:12:13.096100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:12:13.110379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1102/2205] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:12:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:12:17.730401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=57 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:12:23.101496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=28 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:12:34.783991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:12:34.798882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607ab7-401e-4d56-96de-144c1a7a0e86 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:12:34.804984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:12:34.805649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:34.805934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:34.806212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:34.809073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:34.875643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:12:34.878287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473481e-94aa-4ee4-a46c-2d1ed5db7700 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:12:34.888389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1104/2206] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:12:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:12:38.480028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:12:38.500382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459493e-25ee-4ade-a33a-3a73aa709fd4 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:12:38.511041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:12:38.516714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:38.516714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:38.516714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:38.516714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:38.540940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:12:38.542503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:12:38.575656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1103/2207] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:12:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:12:44.883767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=28 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:12:48.558538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:12:53.890684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:53.891271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:53.891271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:53.909194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] MFA Rules not processed for user `8498989e673d445dbc70aa9df55f391b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:12:53.910436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:53.918255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1105/2208] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:12:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:12:54.249279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:12:54.250947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:54.266072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fd790-f211-40de-bbb4-31e7c97550a3 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:12:54.270945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:12:54.272014 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:54.272303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:54.272585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:54.274490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:54.286130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:12:54.287209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:12:54.295808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1104/2209] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:12:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:12:54.729857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:12:54.732618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:54.832155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc754a-34e0-42cc-9391-7766c0e3c4ff None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:12:54.838290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:12:54.838947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:12:54.839235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:12:54.839544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:12:54.841498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:12:54.853749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:12:54.855418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd032876-361d-40d4-9642-fcb0a4aaec48 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:12:54.866753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1106/2210] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:12:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:13:04.292105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:04.861347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=35 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:06.873750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:06.888333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c582e-1e03-4b0f-ae69-baac5d985a83 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:06.900610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:06.901269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:06.901554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:06.901828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:06.903795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:06.921406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:06.923146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:06.935204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1105/2211] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:13:16.929851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:19.603416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:19.604305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:19.604305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:19.623626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] MFA Rules not processed for user `30ac12728e8f4ec9af534612030d75eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:19.624918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:19.640001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1107/2212] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:13:20.006549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:20.016565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e91dd-f014-4f75-a0f9-94d78d17a201 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:20.021628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:20.022322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:20.022615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:20.022922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:20.024897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:20.036161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:20.037614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:20.047222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1106/2213] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:13:29.618012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:30.042588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:34.333522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:34.343407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca028818-8ff9-4a47-8ca2-d911c6d182c9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:34.347074 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:34.347903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:34.351468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:34.351468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:34.351468 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:34.416185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:34.417643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:34.428509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1108/2214] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:13:39.217412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:39.222534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:39.240499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb8493-545e-4246-bcea-7f4c105e933a service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:39.250033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:39.251183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:39.251673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:39.251981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] PATH_INFO: `/v3/users/70d6f15465b84182a28d96d1808a9660` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:39.253451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d6f15465b84182a28d96d1808a9660)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:39.256246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:39.535629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d6f15465b84182a28d96d1808a9660'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:39.536473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d6f15465b84182a28d96d1808a9660'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:39.680270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d6f15465b84182a28d96d1808a9660'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:39.690089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d6f15465b84182a28d96d1808a9660'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:39.712550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2d7f216e-8995-4493-a38a-fb442830035d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d6f15465b84182a28d96d1808a9660'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:39.724266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1107/2215] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:13:39 2022] DELETE /identity/v3/users/70d6f15465b84182a28d96d1808a9660 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:40.316446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:40.328572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458bc3f-fdd6-484c-bb15-906512c6233f service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:40.415625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:40.416472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.416758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.417038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] PATH_INFO: `/v3/projects/b566750533af4d0d96a3d0e0ffbc45ed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.418248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b566750533af4d0d96a3d0e0ffbc45ed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:40.420664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:40.598324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b566750533af4d0d96a3d0e0ffbc45ed', 'request_id': 'req-6d944003-9eeb-4c76-b270-7860b6f195b0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:40.714754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b566750533af4d0d96a3d0e0ffbc45ed', 'request_id': 'req-6d944003-9eeb-4c76-b270-7860b6f195b0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:40.722937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b566750533af4d0d96a3d0e0ffbc45ed', 'request_id': 'req-6d944003-9eeb-4c76-b270-7860b6f195b0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:40.739900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] The token cache is being invalidate because project b566750533af4d0d96a3d0e0ffbc45ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:13:40.740199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:40.742325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6d944003-9eeb-4c76-b270-7860b6f195b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:40.743512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1109/2216] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:13:40 2022] DELETE /identity/v3/projects/b566750533af4d0d96a3d0e0ffbc45ed => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:40.748432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.748945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.749052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.770704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:40.771982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:40.902814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1108/2217] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:40.913434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.913914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.914303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.938526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:40.939774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:40.954514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1110/2218] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:40.959739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:40.977540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed881c3-9a37-40bc-b117-e3d806d04dc1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:40.986821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:40.988038 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:40.988527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:40.993422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:40.993422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:40.993422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:40.995018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.016953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1109/2219] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:13:41.022888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.036641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e41e5-6cdb-420d-8007-87542230c32b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.046030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.047662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.053161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.053161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.053161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.053161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.079895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:41.254285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1111/2220] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:13:41 2022] POST /identity/v3/projects => generated 373 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:41.259833 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.275382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18332a9-0449-4d58-bda9-50170de76ef9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.339803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.340485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.340771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.341051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.341769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.343704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.351277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:41.580094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.588957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1110/2221] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:13:41 2022] POST /identity/v3/users => generated 287 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:13:41.598517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3df38-3ef8-48e8-8f49-c4da2df6c028 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.608411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.609771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.611837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.612657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.614348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.616961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.630754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1112/2222] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:13:41.635930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.648245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340ac4d-34b9-47b6-9852-32fcd029eb26 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.665533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f790d-7852-4589-b277-6863bb6b89ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.666921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f790d-7852-4589-b277-6863bb6b89ed admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.667580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f790d-7852-4589-b277-6863bb6b89ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.668153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f790d-7852-4589-b277-6863bb6b89ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.669813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f790d-7852-4589-b277-6863bb6b89ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.672838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f790d-7852-4589-b277-6863bb6b89ed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.690700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1111/2223] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:13:41.693862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:41.704762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875ad5-7717-4f92-a1bc-6e195bb14cc7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:41.716530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:41.717518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.717805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.718339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] PATH_INFO: `/v3/projects/37dc0134db3d4e52ac4d8d6c16e3f329/users/6b03fe85ce744b38b1ed4d47fabc46ec/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:41.737511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37dc0134db3d4e52ac4d8d6c16e3f329, user_id=6b03fe85ce744b38b1ed4d47fabc46ec, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:41.740542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:41.963545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1113/2224] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:13:41 2022] PUT /identity/v3/projects/37dc0134db3d4e52ac4d8d6c16e3f329/users/6b03fe85ce744b38b1ed4d47fabc46ec/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:41.988528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50382a-2f06-4c6d-951b-c178105366cc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:41.988528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50382a-2f06-4c6d-951b-c178105366cc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:41.988528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50382a-2f06-4c6d-951b-c178105366cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.044646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eb50382a-2f06-4c6d-951b-c178105366cc None None] MFA Rules not processed for user `6b03fe85ce744b38b1ed4d47fabc46ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:42.047163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eb50382a-2f06-4c6d-951b-c178105366cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:42.151982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1112/2225] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:42.159643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:42.161540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:42.289309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d418933-4278-4775-8bd5-6a253deba675 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:42.295579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:42.296266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:42.296738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:42.297021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:42.308583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:42.312832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:42.332975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1114/2226] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:13:45.842603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9bc9c-cac5-494e-a4e7-d9deb2e2e6fb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:45.843284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9bc9c-cac5-494e-a4e7-d9deb2e2e6fb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:45.843284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9bc9c-cac5-494e-a4e7-d9deb2e2e6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:45.868245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-86f9bc9c-cac5-494e-a4e7-d9deb2e2e6fb None None] MFA Rules not processed for user `712089cbbf9a424fa5d4412ced2cdc58`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:45.870004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-86f9bc9c-cac5-494e-a4e7-d9deb2e2e6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:45.959510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1113/2227] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:47.695118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:47.761169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be42c0-5853-4d86-b239-416357a2515d service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:47.768603 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:47.770397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:47.770911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:47.771802 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] PATH_INFO: `/v3/users/6b03fe85ce744b38b1ed4d47fabc46ec` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:47.782404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b03fe85ce744b38b1ed4d47fabc46ec)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:47.782404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:48.055359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:48.062270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b03fe85ce744b38b1ed4d47fabc46ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:48.076158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b03fe85ce744b38b1ed4d47fabc46ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:48.084331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b03fe85ce744b38b1ed4d47fabc46ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:48.195055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b03fe85ce744b38b1ed4d47fabc46ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:48.195786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b03fe85ce744b38b1ed4d47fabc46ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:48.202908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1115/2228] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:13:47 2022] DELETE /identity/v3/users/6b03fe85ce744b38b1ed4d47fabc46ec => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:48.208357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:48.208816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:48.209074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:48.227546 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:48.228869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:48.290961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1114/2229] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:48.302334 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:48.307265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:48.367456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f521-aa70-4d99-819a-6b633b01eb03 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:48.371200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80361ccc-f1c2-4482-bbde-b2b0275057bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:48.371898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80361ccc-f1c2-4482-bbde-b2b0275057bc service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:48.376015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80361ccc-f1c2-4482-bbde-b2b0275057bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:48.376015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80361ccc-f1c2-4482-bbde-b2b0275057bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:48.383835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80361ccc-f1c2-4482-bbde-b2b0275057bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:48.385331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80361ccc-f1c2-4482-bbde-b2b0275057bc service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:48.397501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1116/2230] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:13:48.928714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:48.930281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:48.943829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d95da-8d69-4cf0-85b1-2037e1680c46 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:48.948532 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:48.949438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:48.949720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:48.949994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] PATH_INFO: `/v3/projects/37dc0134db3d4e52ac4d8d6c16e3f329` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:48.951218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37dc0134db3d4e52ac4d8d6c16e3f329)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:48.953189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:48.962381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:49.103739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37dc0134db3d4e52ac4d8d6c16e3f329', 'request_id': 'req-8b9eccef-4d98-42a5-b756-1292aed2932f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:49.110885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37dc0134db3d4e52ac4d8d6c16e3f329', 'request_id': 'req-8b9eccef-4d98-42a5-b756-1292aed2932f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:49.118781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37dc0134db3d4e52ac4d8d6c16e3f329', 'request_id': 'req-8b9eccef-4d98-42a5-b756-1292aed2932f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:49.246877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] The token cache is being invalidate because project 37dc0134db3d4e52ac4d8d6c16e3f329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:13:49.247767 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:49.247981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8b9eccef-4d98-42a5-b756-1292aed2932f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:13:49.253001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1115/2231] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:13:48 2022] DELETE /identity/v3/projects/37dc0134db3d4e52ac4d8d6c16e3f329 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:13:49.255992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.255992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.256539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.275163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:49.276401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:49.339831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1117/2232] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:49.348835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.349129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.349593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.368139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:49.369376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:49.377626 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1116/2233] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:49.383107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:49.393104 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d91a59-82f0-4a63-91cb-de25579db836 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:49.398647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:49.399320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.399640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.399921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.400652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:13:49.400963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:49.402560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:49.416177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1118/2234] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:13:49.420444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:49.430658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e9ccc-1c8e-4a62-858c-c28020b8242c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:49.435590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:49.436235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.436519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.436800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.437490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:49.439792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:49.601122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1117/2235] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:13:49 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:13:49.603209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:49.613622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869113cc-6eae-4624-9504-2356cf4ffb59 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:49.673040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:49.682783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:49.683491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.683787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.684045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.684745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:49.686951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:49.850751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1119/2236] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:13:49 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:49.855269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:49.873003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad80d-4de1-490e-a29a-9c106033cf2b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:49.880803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:49.882185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.882185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.882537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.883880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:49.886698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:49.903931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1118/2237] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:13:49.908058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:49.919284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc36fd-3774-44a2-bece-6f0b346e2663 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:49.927635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d4ae8-3db5-42bf-95b9-e4532522890d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:49.928601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4ae8-3db5-42bf-95b9-e4532522890d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.928886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4ae8-3db5-42bf-95b9-e4532522890d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.929167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d4ae8-3db5-42bf-95b9-e4532522890d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.930127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d4ae8-3db5-42bf-95b9-e4532522890d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:49.931647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d4ae8-3db5-42bf-95b9-e4532522890d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:49.944580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1120/2238] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:13:49.950003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:49.960392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36c01f-4add-4411-80a9-6b9497943067 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:49.965242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:49.966137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:49.966423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:49.966706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] PATH_INFO: `/v3/projects/10fb3bd668b54c71991b6d506b2ebcda/users/f0469b15f0af4c0bb1da588091de054b/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:49.983590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10fb3bd668b54c71991b6d506b2ebcda, user_id=f0469b15f0af4c0bb1da588091de054b, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:49.986256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:50.115735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1119/2239] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:13:49 2022] PUT /identity/v3/projects/10fb3bd668b54c71991b6d506b2ebcda/users/f0469b15f0af4c0bb1da588091de054b/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:50.126012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:50.126297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:50.126572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:50.154578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] MFA Rules not processed for user `f0469b15f0af4c0bb1da588091de054b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:50.156055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:50.230231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1121/2240] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:50.234931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:50.307809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07deedb4-f165-43eb-aa0b-ef4e0f784fd8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:50.313180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:50.314125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:50.314414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:50.314696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:50.315495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:50.318099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:50.499339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1120/2241] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:13:50 2022] POST /identity/v3/projects => generated 373 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:50.503549 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:50.515979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0487e-4920-4ed8-9abc-055e9896ac81 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:50.587850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:50.588657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:50.588943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:50.589224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:50.589926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:50.592009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:50.774111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1122/2242] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:13:50 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:13:50.784919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:50.796171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e741940-458f-43d1-8bb8-1995d2ea2a2b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:50.801086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:50.801797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:50.802079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:50.802357 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:50.803091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:50.804791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:50.816606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1121/2243] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:13:50.823706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:50.839272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029ad0c-80c7-43c9-a98a-7f2e8d4a9c3a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:50.845555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:50.847319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:50.847674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:50.848094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] PATH_INFO: `/v3/projects/c6a400206b9a4eb1a4c27f6862688bf8/users/b4759f98cd7c46188f873440f60a3fa3/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:50.871043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a400206b9a4eb1a4c27f6862688bf8, user_id=b4759f98cd7c46188f873440f60a3fa3, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:50.873782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:51.190493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1123/2244] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:13:50 2022] PUT /identity/v3/projects/c6a400206b9a4eb1a4c27f6862688bf8/users/b4759f98cd7c46188f873440f60a3fa3/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:13:51.196223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:51.211418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb202f-694a-483b-a50b-122be50592d1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:51.215864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:51.294218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75146da8-75c6-4030-b603-f03259ac9029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:51.294895 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75146da8-75c6-4030-b603-f03259ac9029 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:51.295178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75146da8-75c6-4030-b603-f03259ac9029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:51.295611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75146da8-75c6-4030-b603-f03259ac9029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:51.296434 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75146da8-75c6-4030-b603-f03259ac9029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:51.298140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75146da8-75c6-4030-b603-f03259ac9029 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:51.310281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1122/2245] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:13:51.314942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:51.327173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d05850-4369-4a73-9182-382b7d23fe26 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:51.335742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:51.337252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:51.337831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:51.339404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] PATH_INFO: `/v3/projects/c6a400206b9a4eb1a4c27f6862688bf8/users/b4759f98cd7c46188f873440f60a3fa3/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:51.343504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a400206b9a4eb1a4c27f6862688bf8, user_id=b4759f98cd7c46188f873440f60a3fa3, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:51.348272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:51.369203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1124/2246] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:13:51 2022] PUT /identity/v3/projects/c6a400206b9a4eb1a4c27f6862688bf8/users/b4759f98cd7c46188f873440f60a3fa3/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:13:51.380516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:51.381144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:51.383465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:51.414957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] MFA Rules not processed for user `b4759f98cd7c46188f873440f60a3fa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:13:51.416240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:51.520821 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1123/2247] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:13:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:13:51.532843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:51.534598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:13:51.546628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:13:51.600309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e473a4-010d-497e-975a-db884bd299cc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:51.604168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:51.605140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:51.605511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:51.605861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:51.620081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:51.624574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:51.636140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1125/2248] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:13:52.258465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:13:52.269128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68447-0105-4aba-8041-653bcbbf80b9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:13:52.275969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dd150-0ff7-4f5b-b6c7-c132df3f89b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:13:52.276707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dd150-0ff7-4f5b-b6c7-c132df3f89b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:13:52.277096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dd150-0ff7-4f5b-b6c7-c132df3f89b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:13:52.277435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dd150-0ff7-4f5b-b6c7-c132df3f89b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:13:52.355662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8dd150-0ff7-4f5b-b6c7-c132df3f89b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:13:52.357370 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8dd150-0ff7-4f5b-b6c7-c132df3f89b8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:13:52.368304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1124/2249] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:13:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:14:02.362906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:02.544165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=23 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:14:58.568277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:58.636195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03835fd-d8b2-4415-8d22-2f1cddee15c8 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:58.640913 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:58.641783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.642061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.642332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] PATH_INFO: `/v3/users/f0469b15f0af4c0bb1da588091de054b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.643567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0469b15f0af4c0bb1da588091de054b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:58.645572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:58.853524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0469b15f0af4c0bb1da588091de054b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:58.866133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0469b15f0af4c0bb1da588091de054b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:58.873797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0469b15f0af4c0bb1da588091de054b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:58.940412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0469b15f0af4c0bb1da588091de054b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:58.941096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0469b15f0af4c0bb1da588091de054b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:58.948491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1126/2250] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:14:58 2022] DELETE /identity/v3/users/f0469b15f0af4c0bb1da588091de054b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:14:58.953018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:58.953538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:58.953655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:58.971701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:14:58.972931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:59.034666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1125/2251] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:14:59.045495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.046945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:59.116886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8355bb-6622-462e-b0e0-679770953687 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.121181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.121841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.122128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.122472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.132309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.133755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.145064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1127/2252] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:14:59.651678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:14:59.653200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:14:59.667353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34c626-ca65-48dc-af84-f28f3e144905 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:14:59.673962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:14:59.673962 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:14:59.674225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:14:59.674399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] PATH_INFO: `/v3/projects/10fb3bd668b54c71991b6d506b2ebcda` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:14:59.675661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10fb3bd668b54c71991b6d506b2ebcda)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:14:59.677584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:14:59.945096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10fb3bd668b54c71991b6d506b2ebcda', 'request_id': 'req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:59.951671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10fb3bd668b54c71991b6d506b2ebcda', 'request_id': 'req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:14:59.957872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10fb3bd668b54c71991b6d506b2ebcda', 'request_id': 'req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.088878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] The token cache is being invalidate because project 10fb3bd668b54c71991b6d506b2ebcda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:00.089165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.089979 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.091498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1126/2253] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:14:59 2022] DELETE /identity/v3/projects/10fb3bd668b54c71991b6d506b2ebcda => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:00.095521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:00.105837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835aad8-417f-489e-b87d-58f3adf61449 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:00.125094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:00.163974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:00.164858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:00.165146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:00.165455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] PATH_INFO: `/v3/users/b4759f98cd7c46188f873440f60a3fa3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:00.166979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4759f98cd7c46188f873440f60a3fa3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:00.168977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:00.458366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4759f98cd7c46188f873440f60a3fa3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.470753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4759f98cd7c46188f873440f60a3fa3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.479162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4759f98cd7c46188f873440f60a3fa3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.626103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4759f98cd7c46188f873440f60a3fa3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.626734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3e7b40d3-6019-48fa-8f85-360c92d01bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4759f98cd7c46188f873440f60a3fa3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:00.633842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1128/2254] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:00 2022] DELETE /identity/v3/users/b4759f98cd7c46188f873440f60a3fa3 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:01.125809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.128906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:01.147189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfde6d-75d4-441d-bec9-0d1c0c07e1a9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.267086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.268529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.268976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.269322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] PATH_INFO: `/v3/projects/c6a400206b9a4eb1a4c27f6862688bf8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.271041 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a400206b9a4eb1a4c27f6862688bf8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.273310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.405828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:01.439499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a400206b9a4eb1a4c27f6862688bf8', 'request_id': 'req-44f7fea5-8905-496f-a5fd-917b3687ecdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:01.450028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a400206b9a4eb1a4c27f6862688bf8', 'request_id': 'req-44f7fea5-8905-496f-a5fd-917b3687ecdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:01.457095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a400206b9a4eb1a4c27f6862688bf8', 'request_id': 'req-44f7fea5-8905-496f-a5fd-917b3687ecdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:01.542601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] The token cache is being invalidate because project c6a400206b9a4eb1a4c27f6862688bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:01.543028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:01.544059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-44f7fea5-8905-496f-a5fd-917b3687ecdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:01.546910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1127/2255] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:01 2022] DELETE /identity/v3/projects/c6a400206b9a4eb1a4c27f6862688bf8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:01.556425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.557554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.558072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.580251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:01.581769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:01.664621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1129/2256] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:01.673338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.675094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.675094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.701752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:01.703195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:01.711756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1128/2257] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:01.717529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.727613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ca76d-d7c0-43e3-9e13-db902442fb99 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.733167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.733841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.734125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.734401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.735130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:01.735479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.737081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.750553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1130/2258] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:01.755177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.764748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe45ccf-6098-46ca-9c12-1ac79d18bb6c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.769659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.772344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.772344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.772344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.772344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.773733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:01.865802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1129/2259] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:01 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:01.866964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:01.877759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b288-e7bc-4980-9615-e362ad67e4a9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:01.906112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:01.940905 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:01.941622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:01.941909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:01.942133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:01.942826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:01.945224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.133770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1131/2260] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:01 2022] POST /identity/v3/users => generated 282 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:02.138715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.151094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec6860-719e-4ee9-b80f-6d1c09b0e0de admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.155970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.156753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.156977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.157250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.157981 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.159608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.177969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1130/2261] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:02.181379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.192455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f67ab-4f59-40e0-a595-6eb0fc98211b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.198326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1714ffd-fc92-45c8-9bee-8f1e0e2b7613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.199120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714ffd-fc92-45c8-9bee-8f1e0e2b7613 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.199515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714ffd-fc92-45c8-9bee-8f1e0e2b7613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.199899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714ffd-fc92-45c8-9bee-8f1e0e2b7613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.200713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714ffd-fc92-45c8-9bee-8f1e0e2b7613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.202504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714ffd-fc92-45c8-9bee-8f1e0e2b7613 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.219162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1132/2262] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:02.220548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.240721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88305541-d1d6-40fd-bd86-eaeee26563ac admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.248762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.253435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.253435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.253435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] PATH_INFO: `/v3/projects/df55b1e7e99240dcaff38f3cfec569ea/users/6e07fc51a3b64115ac98591d839dbd94/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.271754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df55b1e7e99240dcaff38f3cfec569ea, user_id=6e07fc51a3b64115ac98591d839dbd94, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.276301 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.375459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1131/2263] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:02 2022] PUT /identity/v3/projects/df55b1e7e99240dcaff38f3cfec569ea/users/6e07fc51a3b64115ac98591d839dbd94/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:02.386675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.387077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.387383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.415208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] MFA Rules not processed for user `6e07fc51a3b64115ac98591d839dbd94`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:02.417237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:02.492278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1133/2264] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:02.497003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.574142 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d300f-b4e3-4cc5-a807-d4371955cceb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.578871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.579572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.579855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.580137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.580846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.582758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.779242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1132/2265] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:02 2022] POST /identity/v3/projects => generated 365 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:02.783017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:02.795256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4940a-8454-47f6-8ef4-dc63d195d755 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:02.863463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:02.864152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:02.864440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:02.864722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:02.865423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:02.867402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:02.981799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1134/2266] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:02 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:02.986475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.001310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30260c68-679a-4ce2-982b-6167a6476d09 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.007660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.008344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.008633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.008906 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.010502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.012175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.023942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1133/2267] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:03.028977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.043733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d38b19-4a68-40d9-890e-5796b78e47b3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.048981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.050190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.050714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.051236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] PATH_INFO: `/v3/projects/b29b639af83c469cb56baf51bfee306d/users/c6cba740e60342a789d2f7f06f6c50e1/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.072246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29b639af83c469cb56baf51bfee306d, user_id=c6cba740e60342a789d2f7f06f6c50e1, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.075028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.200593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1135/2268] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:03 2022] PUT /identity/v3/projects/b29b639af83c469cb56baf51bfee306d/users/c6cba740e60342a789d2f7f06f6c50e1/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:03.212365 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.222204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec810d-70a9-4fca-aa8d-809a2249e3f2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.246174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:03.291170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551b368-b519-454e-a456-df55f5d0845c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.291937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551b368-b519-454e-a456-df55f5d0845c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.292106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551b368-b519-454e-a456-df55f5d0845c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.292419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551b368-b519-454e-a456-df55f5d0845c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.293160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551b368-b519-454e-a456-df55f5d0845c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.294937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551b368-b519-454e-a456-df55f5d0845c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.307439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1134/2269] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:03.311557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.317327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:03.323114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205432e-4606-46ee-884c-ef7940ab3d7f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.329630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.330695 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.330829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.331095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] PATH_INFO: `/v3/projects/b29b639af83c469cb56baf51bfee306d/users/c6cba740e60342a789d2f7f06f6c50e1/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.333261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29b639af83c469cb56baf51bfee306d, user_id=c6cba740e60342a789d2f7f06f6c50e1, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.336132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.348710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1136/2270] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:03 2022] PUT /identity/v3/projects/b29b639af83c469cb56baf51bfee306d/users/c6cba740e60342a789d2f7f06f6c50e1/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:03.360020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.360406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.360777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.394660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] MFA Rules not processed for user `c6cba740e60342a789d2f7f06f6c50e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:03.398658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:03.487499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1135/2271] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:03.500995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:03.503329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:03.575131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b573a-085a-4434-bfcf-904e8a623c4e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:03.579841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e61d9a-a788-48b1-9e4b-24d09367d309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:03.580672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e61d9a-a788-48b1-9e4b-24d09367d309 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:03.581108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e61d9a-a788-48b1-9e4b-24d09367d309 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:03.581551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e61d9a-a788-48b1-9e4b-24d09367d309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:03.595492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e61d9a-a788-48b1-9e4b-24d09367d309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:03.598299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e61d9a-a788-48b1-9e4b-24d09367d309 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:03.609146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1137/2272] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:05.204395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.272403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da054c-32bb-432a-a95a-0874135d532e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.277972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.278856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.279135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.279435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] PATH_INFO: `/v3/users/6e07fc51a3b64115ac98591d839dbd94` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.280623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e07fc51a3b64115ac98591d839dbd94)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.282547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.579852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e07fc51a3b64115ac98591d839dbd94'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:05.580521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e07fc51a3b64115ac98591d839dbd94'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:05.582383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:05.689116 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e07fc51a3b64115ac98591d839dbd94'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:05.698027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e07fc51a3b64115ac98591d839dbd94'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:05.714919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e07fc51a3b64115ac98591d839dbd94'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:05.721562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1136/2273] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:05 2022] DELETE /identity/v3/users/6e07fc51a3b64115ac98591d839dbd94 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:05.728222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.729830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.729830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.747055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:05.748325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:05.812276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1138/2274] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:05.822903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:05.824353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:05.887453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f5264-a4e8-4221-87a1-bb1db293807e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:05.891774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:05.892433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:05.892712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:05.892992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:05.902853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:05.904593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:05.914501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1137/2275] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:06.458434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.460020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:06.472997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06f5e1-7289-4905-9c45-8afc42ca22b3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.479731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.480613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.480901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.481193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] PATH_INFO: `/v3/projects/df55b1e7e99240dcaff38f3cfec569ea` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.482543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=df55b1e7e99240dcaff38f3cfec569ea)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.484737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:06.616657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df55b1e7e99240dcaff38f3cfec569ea', 'request_id': 'req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:06.695192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df55b1e7e99240dcaff38f3cfec569ea', 'request_id': 'req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:06.702432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df55b1e7e99240dcaff38f3cfec569ea', 'request_id': 'req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:06.715330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] The token cache is being invalidate because project df55b1e7e99240dcaff38f3cfec569ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:06.715649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:06.716312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-10dfdf5f-b353-4e6a-bc8d-2175247e4eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:06.718471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1139/2276] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:06 2022] DELETE /identity/v3/projects/df55b1e7e99240dcaff38f3cfec569ea => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:06.722373 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:06.733726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b1283-93a9-445b-95f5-404d2db4b714 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:06.796474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:06.797887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:06.797887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:06.797887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] PATH_INFO: `/v3/users/c6cba740e60342a789d2f7f06f6c50e1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:06.799043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6cba740e60342a789d2f7f06f6c50e1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:06.801012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:07.120265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6cba740e60342a789d2f7f06f6c50e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:07.120972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6cba740e60342a789d2f7f06f6c50e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:07.166286 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748658-204f-42a9-9402-b710e592cc6a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.166872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748658-204f-42a9-9402-b710e592cc6a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.167055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748658-204f-42a9-9402-b710e592cc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.185507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-45748658-204f-42a9-9402-b710e592cc6a None None] MFA Rules not processed for user `8498989e673d445dbc70aa9df55f391b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:07.186561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-45748658-204f-42a9-9402-b710e592cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:07.196546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.220354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6cba740e60342a789d2f7f06f6c50e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:07.228272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6cba740e60342a789d2f7f06f6c50e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:07.234000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:07.241234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-538df405-4522-40d8-bac8-0c46871134f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6cba740e60342a789d2f7f06f6c50e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:07.246437 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1140/2277] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:15:07.249392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1138/2278] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:06 2022] DELETE /identity/v3/users/c6cba740e60342a789d2f7f06f6c50e1 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:07.759316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748658-204f-42a9-9402-b710e592cc6a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:07.760829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-45748658-204f-42a9-9402-b710e592cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:07.775826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748658-204f-42a9-9402-b710e592cc6a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:07.838083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:07.839028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:07.839342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:07.839682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] PATH_INFO: `/v3/projects/b29b639af83c469cb56baf51bfee306d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:07.840896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29b639af83c469cb56baf51bfee306d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:07.842795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.001981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29b639af83c469cb56baf51bfee306d', 'request_id': 'req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:08.135030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29b639af83c469cb56baf51bfee306d', 'request_id': 'req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:08.142478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29b639af83c469cb56baf51bfee306d', 'request_id': 'req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:08.153799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] The token cache is being invalidate because project b29b639af83c469cb56baf51bfee306d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:08.154090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:08.154766 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23c5009b-f37b-4fd6-ad44-a4bbdd662c04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:08.156353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1141/2279] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:07 2022] DELETE /identity/v3/projects/b29b639af83c469cb56baf51bfee306d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:08.161500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.162036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.162194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.181931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:08.183150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:08.243630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1139/2280] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:15:08.253185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.253185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.253185 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.272351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:08.273580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:08.282365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1142/2281] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:15:08.286776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.297644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e0a6a-b531-41cd-884a-801a51025f65 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.302346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.303009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.303291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.303594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.304320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:08.304623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.306278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.320539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1140/2282] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:08.323449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.332985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bfffc-a412-48f4-9503-385076886f6b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.338133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.338814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.339101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.339408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.340792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.342668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.479580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1143/2283] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:08 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:08.483924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.494143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e2791-8b2e-437b-85e1-1ca582c7791a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.552409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.552990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.553278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.553557 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.554263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.556212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.707904 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1141/2284] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:08 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:08.711996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.723405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02bedf-881b-4858-b8ce-bd2f32d205a3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.728296 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.728997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.729288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.729570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.730381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.732184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.744433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1144/2285] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:08.748702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.758986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eafd9c-f984-49bf-ac62-f3691437a545 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.764401 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207c73f-22ca-4969-9970-7e5e4999562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.765071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c73f-22ca-4969-9970-7e5e4999562e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.765356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c73f-22ca-4969-9970-7e5e4999562e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.765634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207c73f-22ca-4969-9970-7e5e4999562e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.766414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207c73f-22ca-4969-9970-7e5e4999562e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.768060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207c73f-22ca-4969-9970-7e5e4999562e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:08.779554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1142/2286] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:08.783712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:08.794527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58530325-3699-4f1c-9819-c092f6aa0236 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:08.799219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:08.800175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:08.800470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:08.800733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] PATH_INFO: `/v3/projects/a0861b4f6f1b4f0f838b4c0281a42c1d/users/7db15788dbb54ddf83e971f0b78ae839/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:08.810047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:08.817299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0861b4f6f1b4f0f838b4c0281a42c1d, user_id=7db15788dbb54ddf83e971f0b78ae839, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:08.820435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.074713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1145/2287] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:08 2022] PUT /identity/v3/projects/a0861b4f6f1b4f0f838b4c0281a42c1d/users/7db15788dbb54ddf83e971f0b78ae839/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:09.085958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.086258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.086539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.116236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] MFA Rules not processed for user `7db15788dbb54ddf83e971f0b78ae839`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:09.117463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:09.167111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:09.190368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1143/2288] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:09.199467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.268560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd28718-69cf-4a5c-94c3-c5f678915581 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.274297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.274978 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.275259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.275560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.276273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.278874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.411192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1146/2289] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:09 2022] POST /identity/v3/projects => generated 387 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:09.414971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.427806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17382779-a75c-4804-b3f0-f218944e1769 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.485986 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.486665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.486956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.487237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.487956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.489864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.656020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1144/2290] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:09 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:09.660374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.676060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebaaab-ae79-4818-84fe-ecf5b9458665 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.678631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.679332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.679651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.679933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.680665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.682298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.693427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1147/2291] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:09.697898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.708943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d799-9f75-4765-98b9-0f02aa64f89e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.713798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7004a08-f58d-4475-9fc9-00498ecf4bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.714458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7004a08-f58d-4475-9fc9-00498ecf4bca admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.717773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7004a08-f58d-4475-9fc9-00498ecf4bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.717773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7004a08-f58d-4475-9fc9-00498ecf4bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.717773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7004a08-f58d-4475-9fc9-00498ecf4bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.718042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7004a08-f58d-4475-9fc9-00498ecf4bca admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.728412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1145/2292] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:09.733387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:09.744881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19158b-8b05-4d2a-b930-f39dab4d4279 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:09.751022 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:09.752016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.759342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.759342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] PATH_INFO: `/v3/projects/456ea33207bf4a13a48328dfa7056134/users/0d4c6bf4c9a84953bd64c11909400da4/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.769364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=456ea33207bf4a13a48328dfa7056134, user_id=0d4c6bf4c9a84953bd64c11909400da4, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:09.772057 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:09.872642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1148/2293] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:09 2022] PUT /identity/v3/projects/456ea33207bf4a13a48328dfa7056134/users/0d4c6bf4c9a84953bd64c11909400da4/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:09.883130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:09.883802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:09.884304 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:09.913746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] MFA Rules not processed for user `0d4c6bf4c9a84953bd64c11909400da4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:09.915139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:09.981377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1146/2294] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:09.985626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.051011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ce5ef-4c1b-43a2-a85b-ab621f80f34f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.055798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.056457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.056744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.057024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.057720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.060608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.327119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1149/2295] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:09 2022] POST /identity/v3/projects => generated 383 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:10.331306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.342101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd542ec5-3345-4b29-89ab-9cbb48874b2f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.400729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.401491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.401785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.402063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.402842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.404897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.570440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1147/2296] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:10 2022] POST /identity/v3/users => generated 298 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:10.575510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.585966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef9194-15f6-4ee4-8a3d-eab3771bad5b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.591493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.592176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.592461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.592741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.593008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.593487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.595103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.607306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1150/2297] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:10.612074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.623325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a496f4-faec-467e-aa34-b37306c3c6f1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.628395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.629298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.629584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.629857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] PATH_INFO: `/v3/projects/dfc79152a3b04cdc976097337050b6d5/users/fd4bc301a7d64f169899e28b3a513ad4/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.646868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc79152a3b04cdc976097337050b6d5, user_id=fd4bc301a7d64f169899e28b3a513ad4, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.649554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.787470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1148/2298] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:10 2022] PUT /identity/v3/projects/dfc79152a3b04cdc976097337050b6d5/users/fd4bc301a7d64f169899e28b3a513ad4/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:10.794164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.807250 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2788a3-80df-48b6-9ebe-6d1f9a336cdf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.819035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:10.874960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed5f6a-089a-4494-a26e-86e3ba24aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.875599 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5f6a-089a-4494-a26e-86e3ba24aaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.875889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5f6a-089a-4494-a26e-86e3ba24aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.876173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5f6a-089a-4494-a26e-86e3ba24aaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.876903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed5f6a-089a-4494-a26e-86e3ba24aaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.878539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed5f6a-089a-4494-a26e-86e3ba24aaa2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.889747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1151/2299] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:10.894246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:10.905649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f15e4a-5082-4aa6-b662-52ecca12eace admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:10.910457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:10.911465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.911754 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.912036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] PATH_INFO: `/v3/projects/dfc79152a3b04cdc976097337050b6d5/users/fd4bc301a7d64f169899e28b3a513ad4/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.914006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc79152a3b04cdc976097337050b6d5, user_id=fd4bc301a7d64f169899e28b3a513ad4, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:10.916698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:10.928571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1149/2300] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:10 2022] PUT /identity/v3/projects/dfc79152a3b04cdc976097337050b6d5/users/fd4bc301a7d64f169899e28b3a513ad4/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:10.939742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:10.940036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:10.943984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:10.968728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] MFA Rules not processed for user `fd4bc301a7d64f169899e28b3a513ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:10.969968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:11.046113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1152/2301] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:11.061340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:11.063503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:11.133241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf2923-75a3-4729-bd98-8976eca240bd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:11.137818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:11.138616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:11.138903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:11.139273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:11.149239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:11.150784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:11.161165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1150/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:13.024461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:13.025977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:13.038158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd46b8-2d31-409d-b9b9-9305995d9ac3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:13.042037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:13.042717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:13.043002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:13.043283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:13.108115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:13.109561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:13.120031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1153/2303] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:21.505451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:21.514718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f40d3-dbb7-4acc-9e4c-87042e612509 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:21.519312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe39acb-434c-4694-b65e-509ded13b778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:21.520153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe39acb-434c-4694-b65e-509ded13b778 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:21.520441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe39acb-434c-4694-b65e-509ded13b778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:21.520720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe39acb-434c-4694-b65e-509ded13b778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:21.584956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe39acb-434c-4694-b65e-509ded13b778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:21.586361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe39acb-434c-4694-b65e-509ded13b778 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:21.597082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1151/2304] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:23.115095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:26.810396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:26.876570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae7496-032d-4a68-9742-4570b0963fcf service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:26.882570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:26.883563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:26.883856 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:26.884135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] PATH_INFO: `/v3/users/7db15788dbb54ddf83e971f0b78ae839` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:26.885333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db15788dbb54ddf83e971f0b78ae839)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:26.887244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.199707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db15788dbb54ddf83e971f0b78ae839'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:27.212696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db15788dbb54ddf83e971f0b78ae839'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:27.219438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db15788dbb54ddf83e971f0b78ae839'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:27.296103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db15788dbb54ddf83e971f0b78ae839'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:27.296954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db15788dbb54ddf83e971f0b78ae839'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:27.304894 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1154/2305] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:26 2022] DELETE /identity/v3/users/7db15788dbb54ddf83e971f0b78ae839 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:27.309476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.309945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.314507 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.328817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:27.330709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:27.372202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:27.391757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1152/2306] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:27.402621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:27.404130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:27.465028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3520ce-5686-4d66-8464-0592dec1e48b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:27.468570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:27.469313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:27.469620 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:27.469987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:27.480458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:27.481919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:27.493206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1155/2307] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:28.024841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.026604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:28.038846 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.040786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:28.055745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0419c1-9985-4967-927e-5c47eac635dd service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.080009 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.085780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.085780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.085780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] PATH_INFO: `/v3/projects/a0861b4f6f1b4f0f838b4c0281a42c1d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.085780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0861b4f6f1b4f0f838b4c0281a42c1d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.085780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.109866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.131386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee059fd-d9c5-4682-83d8-09560b447df9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.137124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.137878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.138231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.138579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.140842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:28.153924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.155525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.166920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1153/2308] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:28.272601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0861b4f6f1b4f0f838b4c0281a42c1d', 'request_id': 'req-034f1568-1a9c-499f-8843-fff618192dd2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.393940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0861b4f6f1b4f0f838b4c0281a42c1d', 'request_id': 'req-034f1568-1a9c-499f-8843-fff618192dd2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.402236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0861b4f6f1b4f0f838b4c0281a42c1d', 'request_id': 'req-034f1568-1a9c-499f-8843-fff618192dd2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.416925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] The token cache is being invalidate because project a0861b4f6f1b4f0f838b4c0281a42c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:28.417218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.417870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.419737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1156/2309] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:28 2022] DELETE /identity/v3/projects/a0861b4f6f1b4f0f838b4c0281a42c1d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:28.430211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:28.441826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a910b-1cce-4784-abbe-ec49c695f504 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:28.520337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:28.521226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:28.521513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:28.521795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] PATH_INFO: `/v3/users/0d4c6bf4c9a84953bd64c11909400da4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:28.523491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4c6bf4c9a84953bd64c11909400da4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:28.525428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:28.778881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4c6bf4c9a84953bd64c11909400da4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.779484 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4c6bf4c9a84953bd64c11909400da4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.929594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4c6bf4c9a84953bd64c11909400da4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.937635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4c6bf4c9a84953bd64c11909400da4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.950578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:28.952832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4c6bf4c9a84953bd64c11909400da4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:28.959723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1154/2310] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:28 2022] DELETE /identity/v3/users/0d4c6bf4c9a84953bd64c11909400da4 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:29.458036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.459935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:29.476636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f1568-1a9c-499f-8843-fff618192dd2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.544059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.544992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.545273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.545581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] PATH_INFO: `/v3/projects/456ea33207bf4a13a48328dfa7056134` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.546916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=456ea33207bf4a13a48328dfa7056134)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.549638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:29.706609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '456ea33207bf4a13a48328dfa7056134', 'request_id': 'req-8ec9b018-ea8f-474f-84ea-07c4871571e9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:29.804417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '456ea33207bf4a13a48328dfa7056134', 'request_id': 'req-8ec9b018-ea8f-474f-84ea-07c4871571e9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:29.811943 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '456ea33207bf4a13a48328dfa7056134', 'request_id': 'req-8ec9b018-ea8f-474f-84ea-07c4871571e9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:29.823222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] The token cache is being invalidate because project 456ea33207bf4a13a48328dfa7056134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:29.823547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:29.824236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:29.825786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1157/2311] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:29 2022] DELETE /identity/v3/projects/456ea33207bf4a13a48328dfa7056134 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:29.829975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:29.831607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:29.844425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c5a4a-5477-4ffa-b868-f510d129fbd1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:29.903970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:29.906236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:29.906515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:29.906789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] PATH_INFO: `/v3/users/fd4bc301a7d64f169899e28b3a513ad4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:29.909274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4bc301a7d64f169899e28b3a513ad4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:29.911200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:30.155585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4bc301a7d64f169899e28b3a513ad4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:30.156176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4bc301a7d64f169899e28b3a513ad4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:30.243862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4bc301a7d64f169899e28b3a513ad4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:30.251724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4bc301a7d64f169899e28b3a513ad4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:30.266679 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ef249e01-0459-46e7-94a3-262e31532c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4bc301a7d64f169899e28b3a513ad4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:30.273558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1155/2312] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:29 2022] DELETE /identity/v3/users/fd4bc301a7d64f169899e28b3a513ad4 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:30.773157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:30.774745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:30.790511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9b018-ea8f-474f-84ea-07c4871571e9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:30.847081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:30.873588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:30.874463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:30.879280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:30.879280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] PATH_INFO: `/v3/projects/dfc79152a3b04cdc976097337050b6d5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:30.879280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc79152a3b04cdc976097337050b6d5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:30.879280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.022852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc79152a3b04cdc976097337050b6d5', 'request_id': 'req-1112b6be-5647-4902-ab62-d696b461e8fb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:31.129983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc79152a3b04cdc976097337050b6d5', 'request_id': 'req-1112b6be-5647-4902-ab62-d696b461e8fb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:31.138233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc79152a3b04cdc976097337050b6d5', 'request_id': 'req-1112b6be-5647-4902-ab62-d696b461e8fb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:31.149792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] The token cache is being invalidate because project dfc79152a3b04cdc976097337050b6d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:31.150086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:31.150745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1112b6be-5647-4902-ab62-d696b461e8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:31.152383 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1158/2313] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:30 2022] DELETE /identity/v3/projects/dfc79152a3b04cdc976097337050b6d5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:31.159132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.159715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.159715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.181655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:31.182885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:31.250033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1156/2314] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:31.258573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.258863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.259151 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.278775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:31.279999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:31.290777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1159/2315] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:31.295483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.306751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0f154-e22e-430c-a062-57a240453dbc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.311829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.312515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.312800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.313078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.313850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:31.314160 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.315914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.325719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:31.329374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1157/2316] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:15:31.342887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.353910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed4d02-343f-4bcf-920d-4428f9ab57f4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.358926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.359626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.359910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.360182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.361089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.364165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.497853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1160/2317] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/projects => generated 407 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:31.502915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.512815 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2412e3f-8fbe-449f-9f6a-5fb77a3cddae admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.573710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.574314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.574600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.574880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.575623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.577565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.691242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1158/2318] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/users => generated 304 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:31.697201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.710267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9811949-c183-48fa-b136-838dd0e9238c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.715165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.715903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.716187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.716491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.717225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.718818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.731811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1161/2319] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:31.736769 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.747137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18b556-5c13-4b8c-94d9-9c6d8584048b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.751878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330ce2a-db91-4c30-ad49-fe50d5ec8e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.752549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330ce2a-db91-4c30-ad49-fe50d5ec8e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.752834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330ce2a-db91-4c30-ad49-fe50d5ec8e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.753134 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330ce2a-db91-4c30-ad49-fe50d5ec8e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.753868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330ce2a-db91-4c30-ad49-fe50d5ec8e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.755559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330ce2a-db91-4c30-ad49-fe50d5ec8e62 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.774528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1159/2320] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:15:31.779182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:31.790910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e03c6-bd58-405b-a92a-543f044344c9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:31.795908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:31.796847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.797132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.797408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] PATH_INFO: `/v3/projects/b13497eed6a64e529920f66fc42fab69/users/4de56e46107f4b63b01b4fa295ad9277/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.820126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13497eed6a64e529920f66fc42fab69, user_id=4de56e46107f4b63b01b4fa295ad9277, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:31.823339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:31.901687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1162/2321] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:31 2022] PUT /identity/v3/projects/b13497eed6a64e529920f66fc42fab69/users/4de56e46107f4b63b01b4fa295ad9277/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:31.914643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:31.914935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:31.915221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:31.944560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] MFA Rules not processed for user `4de56e46107f4b63b01b4fa295ad9277`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:31.946120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:32.018519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1160/2322] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:32.022932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.094522 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169f2d9-9bff-4441-8b1f-d35c12fe2aca admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.100447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.101118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.101404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.101709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.102407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.104386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.108891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:32.211990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1163/2323] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:32 2022] POST /identity/v3/projects => generated 409 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:32.216373 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.227612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea31d9d-3e62-4066-9ff3-e1a5412b442a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.289553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.290227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.290515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.290794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.291624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.293445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.417081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:32.425147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1161/2324] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:32 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:32.430098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.444253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac097a4-d90c-4300-b01a-85795e811256 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.449961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.450635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.450920 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.451196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.451963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.453681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.466745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1164/2325] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:32.471920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.491314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0ea6-052a-47a2-a1c7-a609645fa64d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.496409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.497493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.497791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.498071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] PATH_INFO: `/v3/projects/6699c001e7e8463c8f25063b2bbb64d5/users/09e2e2298f4f443a9bb08613eb8e7a17/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.517214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6699c001e7e8463c8f25063b2bbb64d5, user_id=09e2e2298f4f443a9bb08613eb8e7a17, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.523502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.702959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1162/2326] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:32 2022] PUT /identity/v3/projects/6699c001e7e8463c8f25063b2bbb64d5/users/09e2e2298f4f443a9bb08613eb8e7a17/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:32.707085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.718090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378f791-8345-4adb-ba84-c8ef88601013 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.776124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9313b-37c3-4e90-9cbf-4e64d1b181a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.779514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9313b-37c3-4e90-9cbf-4e64d1b181a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.779514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9313b-37c3-4e90-9cbf-4e64d1b181a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.779514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9313b-37c3-4e90-9cbf-4e64d1b181a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.779514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa9313b-37c3-4e90-9cbf-4e64d1b181a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.780530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa9313b-37c3-4e90-9cbf-4e64d1b181a8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.791938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1165/2327] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:32.796159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.809138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e574f7-4567-4747-99c6-68142ef89321 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:32.813796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:32.815455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.815743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.816019 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] PATH_INFO: `/v3/projects/6699c001e7e8463c8f25063b2bbb64d5/users/09e2e2298f4f443a9bb08613eb8e7a17/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.818004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6699c001e7e8463c8f25063b2bbb64d5, user_id=09e2e2298f4f443a9bb08613eb8e7a17, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:32.820934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:32.832395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1163/2328] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:32 2022] PUT /identity/v3/projects/6699c001e7e8463c8f25063b2bbb64d5/users/09e2e2298f4f443a9bb08613eb8e7a17/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:32.843414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:32.843709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:32.843985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:32.873930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] MFA Rules not processed for user `09e2e2298f4f443a9bb08613eb8e7a17`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:32.875155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:32.947912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1166/2329] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:32 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:32.959086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:32.960884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:33.020471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914de873-9aeb-47fb-9805-925833842471 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.024011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.024661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.024947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.025222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.035244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.036787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.046847 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1164/2330] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:33.549399 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:33.559049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce8324-f405-458a-a551-3c441e0c65e3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:33.562681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0020ef-449f-42cb-b66c-3499a4f5102d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:33.563309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0020ef-449f-42cb-b66c-3499a4f5102d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:33.563627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0020ef-449f-42cb-b66c-3499a4f5102d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:33.564510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0020ef-449f-42cb-b66c-3499a4f5102d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:33.624330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:33.627156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0020ef-449f-42cb-b66c-3499a4f5102d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:33.628764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0020ef-449f-42cb-b66c-3499a4f5102d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:33.638186 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1167/2331] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:43.042191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:43.712567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:47.202216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:47.266307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c296a-79f7-4e13-85ff-b01e6412622a service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.272252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.273240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.273522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.273844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] PATH_INFO: `/v3/users/4de56e46107f4b63b01b4fa295ad9277` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.275051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de56e46107f4b63b01b4fa295ad9277)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.277180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:47.625856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de56e46107f4b63b01b4fa295ad9277'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:47.625856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de56e46107f4b63b01b4fa295ad9277'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:47.724722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de56e46107f4b63b01b4fa295ad9277'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:47.734759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de56e46107f4b63b01b4fa295ad9277'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:47.747358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de56e46107f4b63b01b4fa295ad9277'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:47.753879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1165/2332] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:47 2022] DELETE /identity/v3/users/4de56e46107f4b63b01b4fa295ad9277 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:47.759005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.759646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.759646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.777887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:47.779308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:47.850316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1168/2333] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:47.861739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:47.863690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:47.932513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec4d0-7356-49dd-8295-a74616051095 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:47.936840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:47.937658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:47.938051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:47.938457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:47.951760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:47.953504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:47.966926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1166/2334] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:48.568271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:48.569935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:48.589247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e1fdb-3fca-4515-96b1-355a054c98e7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:48.597321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:48.599058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.600163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.600702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] PATH_INFO: `/v3/projects/b13497eed6a64e529920f66fc42fab69` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.603077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13497eed6a64e529920f66fc42fab69)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:48.606869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:48.763924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13497eed6a64e529920f66fc42fab69', 'request_id': 'req-03c9f0a1-8385-47b6-9f14-d39a594c2516'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:48.839999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13497eed6a64e529920f66fc42fab69', 'request_id': 'req-03c9f0a1-8385-47b6-9f14-d39a594c2516'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:48.849143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13497eed6a64e529920f66fc42fab69', 'request_id': 'req-03c9f0a1-8385-47b6-9f14-d39a594c2516'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:48.868070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] The token cache is being invalidate because project b13497eed6a64e529920f66fc42fab69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:48.868367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:48.869180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:48.871184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1169/2335] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:48 2022] DELETE /identity/v3/projects/b13497eed6a64e529920f66fc42fab69 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:48.876577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:48.891720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c697d4-97e8-4ce7-8085-bd4f027b7df7 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:48.916328 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:48.960003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:48.960995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:48.961581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:48.961973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] PATH_INFO: `/v3/users/09e2e2298f4f443a9bb08613eb8e7a17` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:48.963879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e2e2298f4f443a9bb08613eb8e7a17)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:48.965910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:49.408202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e2e2298f4f443a9bb08613eb8e7a17'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:49.408849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e2e2298f4f443a9bb08613eb8e7a17'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:49.667566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e2e2298f4f443a9bb08613eb8e7a17'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:49.676435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e2e2298f4f443a9bb08613eb8e7a17'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:49.690374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5453e7fd-ec2f-4d06-a790-2252e250aa19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e2e2298f4f443a9bb08613eb8e7a17'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:49.698658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1167/2336] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:48 2022] DELETE /identity/v3/users/09e2e2298f4f443a9bb08613eb8e7a17 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:50.290325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.291804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:50.306355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c9f0a1-8385-47b6-9f14-d39a594c2516 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.367842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.368712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.369014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.369270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] PATH_INFO: `/v3/projects/6699c001e7e8463c8f25063b2bbb64d5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.370550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6699c001e7e8463c8f25063b2bbb64d5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.372509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.503803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:50.523087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6699c001e7e8463c8f25063b2bbb64d5', 'request_id': 'req-8bcc5861-a946-496b-bf44-10cf158ab8be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:50.639161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6699c001e7e8463c8f25063b2bbb64d5', 'request_id': 'req-8bcc5861-a946-496b-bf44-10cf158ab8be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:50.647439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6699c001e7e8463c8f25063b2bbb64d5', 'request_id': 'req-8bcc5861-a946-496b-bf44-10cf158ab8be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:50.658285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] The token cache is being invalidate because project 6699c001e7e8463c8f25063b2bbb64d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:50.658613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:50.659265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8bcc5861-a946-496b-bf44-10cf158ab8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:50.660955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1170/2337] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:50 2022] DELETE /identity/v3/projects/6699c001e7e8463c8f25063b2bbb64d5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:50.666270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.666804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.666804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.684984 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:50.686197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:50.750210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1168/2338] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:50.758279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.758484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.758752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.777457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:50.778709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:50.788194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1171/2339] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:50.796055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.806924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9410c260-7d50-4bbd-ab4c-21de226359ce None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.811748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.812421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.812709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.812974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.813700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:50.814007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.815632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:50.828538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1169/2340] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:50.832895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:50.842504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e358e2-2b4f-431a-92fc-9e213c51adfa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:50.847617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:50.848276 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:50.848563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:50.848905 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:50.850226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:50.852341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.035761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1172/2341] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:50 2022] POST /identity/v3/projects => generated 369 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:51.040578 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.050811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615141f-efef-459b-b049-4d9c36a609ec admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.081398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:51.119741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.120687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.121187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.121538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.122237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.124202 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.269915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1170/2342] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:51 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:51.274868 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.288195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd946ad-5946-4bb1-8e9f-3605ae752eec admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.294017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.294744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.295031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.295313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.298346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.298346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.311402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1173/2343] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:51.314487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.326699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21712690-0706-4194-9333-e10f443a8f5e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.332513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17592-63d2-4d23-a807-c87eb2ff48cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.333215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17592-63d2-4d23-a807-c87eb2ff48cb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.333506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17592-63d2-4d23-a807-c87eb2ff48cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.333791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17592-63d2-4d23-a807-c87eb2ff48cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.335270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17592-63d2-4d23-a807-c87eb2ff48cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.337312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17592-63d2-4d23-a807-c87eb2ff48cb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.349090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1171/2344] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:15:51.354518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.365418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e12b2-3133-4cff-806f-bdd66d6cd630 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.370378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.371289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.371609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.371886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] PATH_INFO: `/v3/projects/48e7652d14ed4fc7bb21de385c0c4089/users/ac34ce3c16834b1090fc3b0934e1a170/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.389123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e7652d14ed4fc7bb21de385c0c4089, user_id=ac34ce3c16834b1090fc3b0934e1a170, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.391869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.479775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1174/2345] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:51 2022] PUT /identity/v3/projects/48e7652d14ed4fc7bb21de385c0c4089/users/ac34ce3c16834b1090fc3b0934e1a170/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:51.490349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.490643 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.490969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.518863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] MFA Rules not processed for user `ac34ce3c16834b1090fc3b0934e1a170`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:51.526665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:51.605950 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1172/2346] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:51.618271 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.620036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:51.685002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a580e2b-09a7-4c90-85e6-88566f6dabeb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.688728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b80537-3c2c-4447-a9b7-9303c3c25c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.689425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80537-3c2c-4447-a9b7-9303c3c25c8c service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.689707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80537-3c2c-4447-a9b7-9303c3c25c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.689981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b80537-3c2c-4447-a9b7-9303c3c25c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.700834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b80537-3c2c-4447-a9b7-9303c3c25c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.702261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b80537-3c2c-4447-a9b7-9303c3c25c8c service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.713299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1175/2347] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:51.736394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:51.752095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.752541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.753007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.773460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:51.774903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:51.809992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611111-9eaf-4689-bb66-5b5912f06a05 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:51.814857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:51.815765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:51.816105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:51.816347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] PATH_INFO: `/v3/users/ac34ce3c16834b1090fc3b0934e1a170` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:51.817590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac34ce3c16834b1090fc3b0934e1a170)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:51.820824 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:51.834685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:51.844776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1176/2348] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:15:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:15:52.134952 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:52.136698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:52.221216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dc52e-8cbc-4618-be36-9309a8f374a0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:52.228993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f692411-d909-4b2a-99aa-1e547dab3172 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:52.229622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f692411-d909-4b2a-99aa-1e547dab3172 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:52.229905 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f692411-d909-4b2a-99aa-1e547dab3172 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:52.230177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f692411-d909-4b2a-99aa-1e547dab3172 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:52.242811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f692411-d909-4b2a-99aa-1e547dab3172 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:52.244518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f692411-d909-4b2a-99aa-1e547dab3172 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:52.256493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1177/2349] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:15:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:15:52.264796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:52.273362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac34ce3c16834b1090fc3b0934e1a170'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:52.273362 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac34ce3c16834b1090fc3b0934e1a170'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:52.431256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac34ce3c16834b1090fc3b0934e1a170'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:52.444848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac34ce3c16834b1090fc3b0934e1a170'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:52.466200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac34ce3c16834b1090fc3b0934e1a170'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:52.473933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1173/2350] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:15:51 2022] DELETE /identity/v3/users/ac34ce3c16834b1090fc3b0934e1a170 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:52.478927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:52.484475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:52.484475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:52.506507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:52.508338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:52.601748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1178/2351] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:52.611938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:52.613651 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:52.674441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e280ca6-8cc9-4d27-b3f1-d3d27f623704 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:52.679130 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d1726-2c2a-4ee7-84a4-f726e0258a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:52.679808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d1726-2c2a-4ee7-84a4-f726e0258a5d service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:52.680098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d1726-2c2a-4ee7-84a4-f726e0258a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:52.680379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d1726-2c2a-4ee7-84a4-f726e0258a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:52.690070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d1726-2c2a-4ee7-84a4-f726e0258a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:52.691711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d1726-2c2a-4ee7-84a4-f726e0258a5d service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:52.702673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1174/2352] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:53.392749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:53.395195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:53.414083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8adf0f-5690-495d-af18-f71a584bf15e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:53.422782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:53.431997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.431997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.431997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] PATH_INFO: `/v3/projects/48e7652d14ed4fc7bb21de385c0c4089` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:53.444961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e7652d14ed4fc7bb21de385c0c4089)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:53.452604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:53.753663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e7652d14ed4fc7bb21de385c0c4089', 'request_id': 'req-72756f44-872f-4fed-b675-083a24fa7c60'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:53.926120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e7652d14ed4fc7bb21de385c0c4089', 'request_id': 'req-72756f44-872f-4fed-b675-083a24fa7c60'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:53.948636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e7652d14ed4fc7bb21de385c0c4089', 'request_id': 'req-72756f44-872f-4fed-b675-083a24fa7c60'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:53.974463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] The token cache is being invalidate because project 48e7652d14ed4fc7bb21de385c0c4089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:15:53.976409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:53.977426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-72756f44-872f-4fed-b675-083a24fa7c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:15:53.981086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1179/2353] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:15:53 2022] DELETE /identity/v3/projects/48e7652d14ed4fc7bb21de385c0c4089 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:15:53.988468 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:53.989397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:53.991632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.033368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:54.050722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:54.222740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1175/2354] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:54.335758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.342290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.347188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.397414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:54.400390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:54.416284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1180/2355] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:54.421963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.439630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6775df-0a1f-403e-bfc9-826e5c654eac None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.449758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.450842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.452941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.452941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.453515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:15:54.453680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.456417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.483114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1176/2356] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:15:54.490459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.503780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.508745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5945f99-f61f-48ed-a9a5-2dcdab729a0e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.516709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.517854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.525617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.525617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.525617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.525617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:54.694050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1181/2357] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:15:54 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:15:54.699768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:54.721325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c79e6f-dc07-4e99-9739-7418f2532026 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:54.813375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:15:54.837960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:54.838805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:54.839016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:54.839296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:54.840061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:54.842078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.095213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1177/2358] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:15:54 2022] POST /identity/v3/users => generated 288 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:15:55.101606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.133992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe4c69-73d8-41e7-b43f-ea72dab59eb3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.142187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.143173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.143725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.144275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.145806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.148984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.164788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1182/2359] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:15:55.169012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.181697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb7df-4daa-4708-a270-d92c1dbeb6a6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.188382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa048a2-ed09-43e5-a782-ded4cbf31777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.189283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa048a2-ed09-43e5-a782-ded4cbf31777 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.189675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa048a2-ed09-43e5-a782-ded4cbf31777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.190092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa048a2-ed09-43e5-a782-ded4cbf31777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.191305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa048a2-ed09-43e5-a782-ded4cbf31777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.199857 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa048a2-ed09-43e5-a782-ded4cbf31777 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.210256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1178/2360] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:15:55.215516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.233684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323740e-62ea-4ae2-bcb0-d2ee1e4ea886 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.246576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.253898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.253898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.253898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] PATH_INFO: `/v3/projects/68570b43d03d4135baa402dbf5528f0c/users/816a72a7849745f0974e03c400b1e585/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.268711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68570b43d03d4135baa402dbf5528f0c, user_id=816a72a7849745f0974e03c400b1e585, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.272550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.423949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1183/2361] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:15:55 2022] PUT /identity/v3/projects/68570b43d03d4135baa402dbf5528f0c/users/816a72a7849745f0974e03c400b1e585/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:15:55.440828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d749a7-12be-4652-a757-83504195ea2f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.441899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d749a7-12be-4652-a757-83504195ea2f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.448450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d749a7-12be-4652-a757-83504195ea2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.498043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-01d749a7-12be-4652-a757-83504195ea2f None None] MFA Rules not processed for user `816a72a7849745f0974e03c400b1e585`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:15:55.500280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-01d749a7-12be-4652-a757-83504195ea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:55.634591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1179/2362] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:15:55.658651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:55.658651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:55.770109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d289b6-aa1d-4f2b-b4b3-20dc964ad54c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:55.775302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:55.776085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:55.776372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:55.776657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:55.786741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:55.788401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:55.815778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1184/2363] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:15:56.225740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d749a7-12be-4652-a757-83504195ea2f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:15:56.227282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-01d749a7-12be-4652-a757-83504195ea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:56.306238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d749a7-12be-4652-a757-83504195ea2f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:15:56.312166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:15:56.312946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:15:56.313220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:15:56.313493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:15:56.315570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:15:56.448149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:15:56.448149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:15:56.448149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1180/2364] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:15:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:05.795177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:06.429454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:08.073902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:08.076683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:08.089901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31145f7b-3a03-437a-8d2b-f4c59011b55f service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:08.093805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:08.094483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:08.094786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:08.095093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:08.097073 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:08.178398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:08.181844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:08.194803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1185/2365] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:13.146729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:13.149048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:13.285885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3581823-3e27-4e9a-970f-ff1480ba6060 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:13.296814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:13.297708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:13.298222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:13.298551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:13.300829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:13.316781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:13.319677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:13.339210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1181/2366] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:18.189458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=30 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:19.011842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.013416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:19.030084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058d730-64b9-4c78-bd77-7721d588f07a service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.037967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.038807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.039091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.039400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.041577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:19.059474 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.062481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.078803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1186/2367] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:19.323603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:19.325892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:19.350902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547d55-ab03-4601-a2cf-b7dd8fc5d0b2 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:19.358190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:19.359081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:19.359378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:19.359668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] PATH_INFO: `/v3/users/2311f0b1e7074500a6964504a7aee218` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:19.361009 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2311f0b1e7074500a6964504a7aee218)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:19.363059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:19.730495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2311f0b1e7074500a6964504a7aee218'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:19.731198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2311f0b1e7074500a6964504a7aee218'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:19.839720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2311f0b1e7074500a6964504a7aee218'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:19.853480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2311f0b1e7074500a6964504a7aee218'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:19.873166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2311f0b1e7074500a6964504a7aee218'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:19.893251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1182/2368] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:19 2022] DELETE /identity/v3/users/2311f0b1e7074500a6964504a7aee218 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:20.404799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:20.434537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c47cc-d0cd-46fa-aad6-82c92690a1f5 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:20.537126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:20.538605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:20.538907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:20.539190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] PATH_INFO: `/v3/projects/bdb225966010445785ba6d65c068176c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:20.540583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb225966010445785ba6d65c068176c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:20.542509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:20.733692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb225966010445785ba6d65c068176c', 'request_id': 'req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.142382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb225966010445785ba6d65c068176c', 'request_id': 'req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.151746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb225966010445785ba6d65c068176c', 'request_id': 'req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.164656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] The token cache is being invalidate because project bdb225966010445785ba6d65c068176c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:21.164955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.165657 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.167469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1187/2369] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:20 2022] DELETE /identity/v3/projects/bdb225966010445785ba6d65c068176c => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:21.171635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:21.173155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:21.190638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f2b32-f716-4eba-9d1d-e18db954efa8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:21.262969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:21.264264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:21.265338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:21.265584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] PATH_INFO: `/v3/users/ec1683d81b244667bf39bb2a53c73838` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:21.267244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1683d81b244667bf39bb2a53c73838)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:21.269344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:21.283733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:21.479094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1683d81b244667bf39bb2a53c73838'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.479919 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1683d81b244667bf39bb2a53c73838'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.623105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1683d81b244667bf39bb2a53c73838'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.632017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1683d81b244667bf39bb2a53c73838'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.644941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bcf6050-b8f7-4aaf-a83c-f4f6804d4f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1683d81b244667bf39bb2a53c73838'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:21.653711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1183/2370] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:21 2022] DELETE /identity/v3/users/ec1683d81b244667bf39bb2a53c73838 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:22.059387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:22.060960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:22.076895 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66688e5-4bed-4d1e-ab82-01d85bca6d2d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:22.161295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:22.163476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.163476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.163747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] PATH_INFO: `/v3/projects/fdb92af186e84f6eb89fb15d292b86ed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.165066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb92af186e84f6eb89fb15d292b86ed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:22.168145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:22.191163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:22.299873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb92af186e84f6eb89fb15d292b86ed', 'request_id': 'req-11fdf67b-fc96-4577-b778-30a4e6f512a4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:22.377209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb92af186e84f6eb89fb15d292b86ed', 'request_id': 'req-11fdf67b-fc96-4577-b778-30a4e6f512a4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:22.386058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb92af186e84f6eb89fb15d292b86ed', 'request_id': 'req-11fdf67b-fc96-4577-b778-30a4e6f512a4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:22.397934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] The token cache is being invalidate because project fdb92af186e84f6eb89fb15d292b86ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:22.398209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:22.399171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-11fdf67b-fc96-4577-b778-30a4e6f512a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:22.401352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1188/2371] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:22 2022] DELETE /identity/v3/projects/fdb92af186e84f6eb89fb15d292b86ed => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:16:22.407005 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.407825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.407825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.428755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:22.430006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:22.506930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1184/2372] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:22.516749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.517065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.517315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.537100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:22.540801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:22.547839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1189/2373] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:22.553680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:22.565074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27f3ef-c267-47ef-a5f3-6a1661354b13 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:22.570351 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:22.571122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.571504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.571875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.572703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:22.573327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:22.575264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:22.591312 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1185/2374] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:22.598247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:22.615799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624c8d6-010c-489b-bb21-4a0151153b70 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:22.624289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:22.625558 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.626063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.626543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.627789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:22.630513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:22.824981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1190/2375] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:22 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:22.840844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:22.846068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225e7b4-f7e5-41aa-be65-3e010923a927 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:22.920156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:22.920873 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:22.921136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:22.921417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:22.922119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:22.924084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:23.164331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:23.174234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1186/2376] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:22 2022] POST /identity/v3/users => generated 289 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:23.180565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:23.199127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d2324-19d2-4fd0-8d15-abd444a3d527 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:23.207028 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:23.208099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:23.208570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:23.209030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:23.210221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:23.213511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:23.231847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1191/2377] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:23.237267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:23.257452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11fdff-d469-4d5b-a4b6-f4eef957a659 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:23.264306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf708-81de-45c1-a148-f341b5eeab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:23.265491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf708-81de-45c1-a148-f341b5eeab8d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:23.266016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf708-81de-45c1-a148-f341b5eeab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:23.266540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf708-81de-45c1-a148-f341b5eeab8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:23.268096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf708-81de-45c1-a148-f341b5eeab8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:23.271489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf708-81de-45c1-a148-f341b5eeab8d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:23.292372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1187/2378] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:23.296803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:23.311984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cbe58-2c36-4b16-8722-60a079a3047f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:23.317371 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:23.318597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:23.318597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:23.318597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] PATH_INFO: `/v3/projects/bff5c01fd0ab4348b4704765f0195d50/users/f8cc86b384df4223a800838a3d386d3f/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:23.339071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff5c01fd0ab4348b4704765f0195d50, user_id=f8cc86b384df4223a800838a3d386d3f, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:23.341979 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:23.609170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1192/2379] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:23 2022] PUT /identity/v3/projects/bff5c01fd0ab4348b4704765f0195d50/users/f8cc86b384df4223a800838a3d386d3f/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:23.614746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:23.615451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:23.615451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:23.637590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:23.639230 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:23.717588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1188/2380] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:23.728783 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:23.731394 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:23.809676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0793e-ce92-402d-9bb8-db283e9e3095 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:23.813938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:23.815321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:23.815321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:23.815692 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:23.830085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:23.831642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:23.843572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1193/2381] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:26.617960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:26.629758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623d2347-cb84-45d2-a537-e06082a1789d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:26.635100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:26.635795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:26.636260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:26.636696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:26.639141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:26.664839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:26.668504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:26.764647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:26.775168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfdbbc-9cd9-4ff6-bd64-f409f992e06e service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:26.778413 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:26.780956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:26.783131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:26.784096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:26.784881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:26.785382 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:26.789509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:26.798640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1189/2382] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:26.813065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:26.814984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:26.833430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1194/2383] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:16:27.292693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:27.302049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:27.336900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a1708-d1f0-425a-bb74-e0a02181109f service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:27.347473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:27.348494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:27.349294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:27.350206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:27.358570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:27.388524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:27.392980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:27.418420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1190/2384] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:27.553041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:27.567096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27217382-c29a-43ff-99c2-889ca816d7e6 service glance] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:27.571560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:27.572215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:27.572499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:27.572779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:27.574951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:27.590480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:27.591988 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c77e33-7dfc-4627-a6a3-d3dab3f7359c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:27.608732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1195/2385] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:28.061804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:28.072995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f712d-6265-4cf2-8cf9-cc25fb7ca5be service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:28.080408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:28.081098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:28.081393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:28.081675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:28.083727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:28.100400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:28.102178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:28.120801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1191/2386] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:32.341993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284499a6-ce29-4676-bb48-0d5b61833ead None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:32.342527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284499a6-ce29-4676-bb48-0d5b61833ead None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:32.342527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284499a6-ce29-4676-bb48-0d5b61833ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:32.373605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-284499a6-ce29-4676-bb48-0d5b61833ead None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:32.379445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-284499a6-ce29-4676-bb48-0d5b61833ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:32.512942 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1196/2387] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:16:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:32.527875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:32.530940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:32.656508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:32.688455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a4656-cce4-45a2-a566-b4a26a633b38 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:32.696033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:32.697044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:32.697550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:32.697980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:32.708183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:32.709973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:32.722237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1192/2388] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:16:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:16:33.270785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:33.271122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:33.272246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:33.339840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:33.343967 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:33.351299 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] MFA Rules not processed for user `f8cc86b384df4223a800838a3d386d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:33.355790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:33.384764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a6aa-63f3-4ed5-bff6-cfec45e2f3fe service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:33.401561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:33.406955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:33.406955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:33.406955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] PATH_INFO: `/v3/users/816a72a7849745f0974e03c400b1e585` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:33.406955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=816a72a7849745f0974e03c400b1e585)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:33.416481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:33.569379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1197/2389] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:33.571581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:33.595429 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e3ab5-6a49-4e0b-9ef8-e2379b883328 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:33.613438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:33.614212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:33.614524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:33.615014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:33.616385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:33.619135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:33.789884 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816a72a7849745f0974e03c400b1e585'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:33.791585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816a72a7849745f0974e03c400b1e585'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:33.810990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1198/2390] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:33 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:33.816603 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:33.914907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816a72a7849745f0974e03c400b1e585'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:33.929296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816a72a7849745f0974e03c400b1e585'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:33.992731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-88799b3e-e8fb-4c91-a488-45c0b9bd1ca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816a72a7849745f0974e03c400b1e585'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:34.015946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1193/2391] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:33 2022] DELETE /identity/v3/users/816a72a7849745f0974e03c400b1e585 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:34.015946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.015946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.015946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.051892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:34.056098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:34.056098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:34.093236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f6862-2a96-497e-93fe-9a3a43c475fa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.229646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.231500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.231811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.232852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.233804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.237011 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.264852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1194/2392] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:34.279234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.281259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:34.376308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1199/2393] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:33 2022] POST /identity/v3/users => generated 295 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:34.383698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.404082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:34.409433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:34.416088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419652a2-04e5-4b02-9e1f-3f7fd59a4bd8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.421003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.421969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.423091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.423091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.438358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.440451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.451909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147f2a9-8a08-47c8-9c74-776f5b6f6106 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.455119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1195/2394] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:34.462632 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.463746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.464220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.464658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.465683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.467975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.482781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1200/2395] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:16:34.488190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.501481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d585c-6147-4a78-a179-383e48a30d39 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.508148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.509608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.510218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.510711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] PATH_INFO: `/v3/projects/ec8d64f5e76a4b28a4583aca81f79e97/users/fc608f0b89744843a231d2b6365d257d/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.543195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8d64f5e76a4b28a4583aca81f79e97, user_id=fc608f0b89744843a231d2b6365d257d, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.543195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.716939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1196/2396] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:34 2022] PUT /identity/v3/projects/ec8d64f5e76a4b28a4583aca81f79e97/users/fc608f0b89744843a231d2b6365d257d/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:34.722706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.746006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9019d43-73aa-4344-8b33-f9ea863b9917 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:34.903230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:34.904231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:34.904587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:34.905134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:34.911279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:34.916171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:34.949598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1201/2397] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:34.950419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:34.977259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae2ac-ade2-45b4-9209-75a048574489 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:35.004689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a31ff-2b7c-4c71-a2f9-a906b4420cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:35.009062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a31ff-2b7c-4c71-a2f9-a906b4420cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:35.009607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a31ff-2b7c-4c71-a2f9-a906b4420cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:35.010478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a31ff-2b7c-4c71-a2f9-a906b4420cf8 admin admin] PATH_INFO: `/v3/projects/ec8d64f5e76a4b28a4583aca81f79e97/users/fc608f0b89744843a231d2b6365d257d/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:35.020372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a31ff-2b7c-4c71-a2f9-a906b4420cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8d64f5e76a4b28a4583aca81f79e97, user_id=fc608f0b89744843a231d2b6365d257d, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:35.034100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a31ff-2b7c-4c71-a2f9-a906b4420cf8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:35.064451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1197/2398] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:34 2022] PUT /identity/v3/projects/ec8d64f5e76a4b28a4583aca81f79e97/users/fc608f0b89744843a231d2b6365d257d/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:35.970310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:35.972977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:36.016811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff910f-a6c1-455a-8117-02d80786f971 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:36.153480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:36.154428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:36.154685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:36.154959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] PATH_INFO: `/v3/projects/68570b43d03d4135baa402dbf5528f0c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:36.156289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68570b43d03d4135baa402dbf5528f0c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:36.160819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:36.213454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:36.417586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68570b43d03d4135baa402dbf5528f0c', 'request_id': 'req-4087368c-94a0-4c72-8313-8a9cf18774d9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:36.589424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68570b43d03d4135baa402dbf5528f0c', 'request_id': 'req-4087368c-94a0-4c72-8313-8a9cf18774d9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:36.606450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68570b43d03d4135baa402dbf5528f0c', 'request_id': 'req-4087368c-94a0-4c72-8313-8a9cf18774d9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:36.632021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] The token cache is being invalidate because project 68570b43d03d4135baa402dbf5528f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:36.632983 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:36.634180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4087368c-94a0-4c72-8313-8a9cf18774d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:36.643357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1202/2399] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:35 2022] DELETE /identity/v3/projects/68570b43d03d4135baa402dbf5528f0c => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:36.644995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:36.645942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:36.646538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:36.682421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:36.684261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:36.963124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:36.963873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:36.964397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:36.980505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1198/2400] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:37.040740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:37.044616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:37.091422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1203/2401] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:37.094870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:37.109682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a6ee-c03d-4476-8636-ff9222ba8f9f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:37.117228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:37.121090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:37.121090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:37.121090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:37.121090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:37.121090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:37.122470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:37.152202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:37.179898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1199/2402] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:37.182209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204dc2d6-c4c0-48a8-bc1f-e004cdeb5298 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:37.202040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:37.202984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:37.203446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:37.206641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:37.206641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:37.208999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:37.837006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1204/2403] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:37 2022] POST /identity/v3/projects => generated 385 bytes in 682 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:37.838980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:37.861937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784f5c7-a938-4f18-8e27-7cd88a061feb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:38.025536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:38.026496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:38.026876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:38.027309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:38.028387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:38.030852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:38.229989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1200/2404] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:37 2022] POST /identity/v3/users => generated 293 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:38.231440 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:38.262923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd25991-b525-4c48-8493-5531d336b3d4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:38.268035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:38.270058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:38.270058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:38.270058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:38.270058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:38.275210 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:38.302896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:38.319696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1205/2405] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:38.347101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626b9db-e1dd-4257-84ef-8c5770a8d40f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:38.354347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0ec3e-7b92-47b7-b3ba-38b63f203053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:38.358171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0ec3e-7b92-47b7-b3ba-38b63f203053 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:38.358708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0ec3e-7b92-47b7-b3ba-38b63f203053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:38.359209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0ec3e-7b92-47b7-b3ba-38b63f203053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:38.364501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0ec3e-7b92-47b7-b3ba-38b63f203053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:38.367170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0ec3e-7b92-47b7-b3ba-38b63f203053 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:38.397487 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1201/2406] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:38.411613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:38.434837 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a5f54-77d6-4f7e-813c-60c4e5344226 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:38.478491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:38.479693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:38.480745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:38.481204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] PATH_INFO: `/v3/projects/e1a929467e8641eda2fcbb95cd157e57/users/5d1510e90ab84fa58cb9fd93583490fe/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:38.507676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a929467e8641eda2fcbb95cd157e57, user_id=5d1510e90ab84fa58cb9fd93583490fe, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:38.511747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:38.675031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1206/2407] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:38 2022] PUT /identity/v3/projects/e1a929467e8641eda2fcbb95cd157e57/users/5d1510e90ab84fa58cb9fd93583490fe/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:38.698191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:38.698863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:38.699296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:38.766538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] MFA Rules not processed for user `5d1510e90ab84fa58cb9fd93583490fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:38.794433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:38.794433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:38.908682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1202/2408] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:38.916996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:39.063769 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38633970-be40-4de5-a37f-7566ee537655 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:39.074080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:39.074916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:39.075303 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:39.075712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:39.077460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:39.081485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:39.290960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:39.306838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1207/2409] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:38 2022] POST /identity/v3/projects => generated 383 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:39.368785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a3561-62c5-4163-9806-34f952a0c46e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:39.696028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:39.698021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:39.698511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:39.699282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:39.705528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:39.710671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:39.885854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1203/2410] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:39 2022] POST /identity/v3/users => generated 292 bytes in 593 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:39.887359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:39.905273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1a176-0d06-4895-a8f6-993b79cffec7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:39.920568 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:39.921841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:39.922557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:39.923118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:39.924875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:39.926935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:39.968229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1208/2411] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:16:39.970140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:39.986104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f8a77-a521-4621-b263-57c9338299b3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.001575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b6d65-815a-4c65-aea5-eea8254c7b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.002694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b6d65-815a-4c65-aea5-eea8254c7b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.003203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b6d65-815a-4c65-aea5-eea8254c7b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.003921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b6d65-815a-4c65-aea5-eea8254c7b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.007215 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b6d65-815a-4c65-aea5-eea8254c7b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.009481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b6d65-815a-4c65-aea5-eea8254c7b8f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.026780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1204/2412] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:40.027500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.045618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5eee2-0078-4160-a3bb-1507b45c20a9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.058368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.061597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.061597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.061597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] PATH_INFO: `/v3/projects/44c9ff8b45d14e9fb51eeb297f55d821/users/b44bd9877a5048af939404b75ec85c92/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.093890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:40.112095 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c9ff8b45d14e9fb51eeb297f55d821, user_id=b44bd9877a5048af939404b75ec85c92, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.116371 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.242311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1209/2413] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:40 2022] PUT /identity/v3/projects/44c9ff8b45d14e9fb51eeb297f55d821/users/b44bd9877a5048af939404b75ec85c92/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:40.257696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269959-751c-4410-9654-ab84c9202e2c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.258360 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269959-751c-4410-9654-ab84c9202e2c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.259008 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269959-751c-4410-9654-ab84c9202e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.301400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-47269959-751c-4410-9654-ab84c9202e2c None None] MFA Rules not processed for user `b44bd9877a5048af939404b75ec85c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:40.303477 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-47269959-751c-4410-9654-ab84c9202e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:40.448964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1205/2414] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:40.454236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.581930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b453a6c-4270-4721-b346-472b0a1f93c7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.595514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:40.597646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:40.599229 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:40.599448 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:40.602521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:40.602521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:40.810772 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1210/2415] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:40 2022] POST /identity/v3/projects => generated 385 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:40.823419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47269959-751c-4410-9654-ab84c9202e2c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:40.848530 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47269959-751c-4410-9654-ab84c9202e2c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:40.910992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:41.125108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.125941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.126232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.129935 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.134025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.138744 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:41.142206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:41.142872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:41.321094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae2eac-b9a6-47a5-81e1-037b319c5fd9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:41.335165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.337037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.337037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.337037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.340728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:41.394479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1206/2416] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:40 2022] POST /identity/v3/users => generated 299 bytes in 574 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:41.398367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:41.425709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.429083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:41.437990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71000a-5c1f-45a0-9abb-2c13ad6f7c9a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:41.457137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1211/2417] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:41.465616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.466964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.467963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.468285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.469164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.471271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:41.500470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1207/2418] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:41.506924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:41.521025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3f521-1b7c-48fa-8e64-ad473d10c200 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:41.538374 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.539345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.539681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.539977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] PATH_INFO: `/v3/projects/843dcb5449ab4f8dba246b9377d939dd/users/fbef57c892ce451d9b950b4d2f20d51c/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.581532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843dcb5449ab4f8dba246b9377d939dd, user_id=fbef57c892ce451d9b950b4d2f20d51c, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.584970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:41.707088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1212/2419] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:41 2022] PUT /identity/v3/projects/843dcb5449ab4f8dba246b9377d939dd/users/fbef57c892ce451d9b950b4d2f20d51c/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:41.714243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:41.742233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724aca9-ac1a-4c86-b178-bc9c13e76bcb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:41.908992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07d6f2-7476-4bae-aae4-fbe6586abe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.910127 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07d6f2-7476-4bae-aae4-fbe6586abe4c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.910727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07d6f2-7476-4bae-aae4-fbe6586abe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.911647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07d6f2-7476-4bae-aae4-fbe6586abe4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.914403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07d6f2-7476-4bae-aae4-fbe6586abe4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.916648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07d6f2-7476-4bae-aae4-fbe6586abe4c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:41.933021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1208/2420] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:16:41.937125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:41.956030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be820-2a79-4b94-8fd9-ad3b34298144 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:41.962078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:41.966613 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:41.967517 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:41.968316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] PATH_INFO: `/v3/projects/843dcb5449ab4f8dba246b9377d939dd/users/fbef57c892ce451d9b950b4d2f20d51c/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:41.970993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843dcb5449ab4f8dba246b9377d939dd, user_id=fbef57c892ce451d9b950b4d2f20d51c, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:41.977466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:42.002986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1213/2421] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:41 2022] PUT /identity/v3/projects/843dcb5449ab4f8dba246b9377d939dd/users/fbef57c892ce451d9b950b4d2f20d51c/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:42.016025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:42.016671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:42.017141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:42.085111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] MFA Rules not processed for user `fbef57c892ce451d9b950b4d2f20d51c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:42.087924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:42.235139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1209/2422] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:42.268481 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:42.276438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:42.342545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:42.390466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95492a2b-f317-4fde-b884-4dbafeaf7071 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:42.405655 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:42.406992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:42.407575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:42.408206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:42.576179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:42.579923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:42.608612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1214/2423] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:45.022433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:45.023974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:45.102595 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:45.267290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b687871-cfaa-4ac7-aa9a-21b67c7d17c6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:45.267290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:45.267290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:45.267290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:45.268035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:45.268035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:45.464969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:45.470327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909917-afbb-42cb-bca3-ce726cdf46b9 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:45.551837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1210/2424] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:47.706398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:47.719351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474c35b-2af6-44ae-b485-8cc520e47d14 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:47.738206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:47.739818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:47.741377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:47.742424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:47.918647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:47.921944 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:47.957455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1215/2425] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:48.019665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:48.020246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:48.020246 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:48.058208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] MFA Rules not processed for user `fc608f0b89744843a231d2b6365d257d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:48.060072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:48.172197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1211/2426] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:48.191207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:48.213089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e128c-915f-4c67-9004-fd5df05cfbad service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:48.220637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:48.221309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:48.221636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:48.221919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:48.226278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:48.360434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:48.362202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:48.375806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1216/2427] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:48.676631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:48.678659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:48.699858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:48.772255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477a2d-1598-454c-901f-cb3bb7f8ef59 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:48.777253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:48.778267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:48.778597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:48.778948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] PATH_INFO: `/v3/users/712089cbbf9a424fa5d4412ced2cdc58` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:48.794591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=712089cbbf9a424fa5d4412ced2cdc58)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:48.796659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:49.097300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712089cbbf9a424fa5d4412ced2cdc58'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:49.098852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712089cbbf9a424fa5d4412ced2cdc58'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:49.253013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712089cbbf9a424fa5d4412ced2cdc58'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:49.267133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712089cbbf9a424fa5d4412ced2cdc58'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:49.284608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d171546-c500-4a83-bd2d-aa649a759e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712089cbbf9a424fa5d4412ced2cdc58'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:49.294138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1212/2428] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:48 2022] DELETE /identity/v3/users/712089cbbf9a424fa5d4412ced2cdc58 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:49.721407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:49.738652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98211da2-c740-40c5-b1e3-7b10be40d416 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:49.774040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:49.809425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:49.810800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:49.811436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:49.811719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] PATH_INFO: `/v3/projects/14b0926322074e2da25ddaae611d89d4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:49.826370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b0926322074e2da25ddaae611d89d4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:49.831730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:49.975201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b0926322074e2da25ddaae611d89d4', 'request_id': 'req-dbe81b69-6336-4812-b683-ddb1a828b8e1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:50.075132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b0926322074e2da25ddaae611d89d4', 'request_id': 'req-dbe81b69-6336-4812-b683-ddb1a828b8e1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:50.085976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b0926322074e2da25ddaae611d89d4', 'request_id': 'req-dbe81b69-6336-4812-b683-ddb1a828b8e1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:50.107202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] The token cache is being invalidate because project 14b0926322074e2da25ddaae611d89d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:50.107854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:50.121040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:50.121040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1217/2429] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:49 2022] DELETE /identity/v3/projects/14b0926322074e2da25ddaae611d89d4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:50.122327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1965d-9def-42e2-904f-ff5b8086c90c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.122805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1965d-9def-42e2-904f-ff5b8086c90c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.122910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1965d-9def-42e2-904f-ff5b8086c90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.144006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bef1965d-9def-42e2-904f-ff5b8086c90c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:50.145694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bef1965d-9def-42e2-904f-ff5b8086c90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:50.282873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.282873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:50.291653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1213/2430] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:50.301513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.301886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.302218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.329653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:50.331920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:50.352715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1214/2431] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:50.355683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.368910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb80392-8410-44c7-99f0-599e211e7f6d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.370496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81b69-6336-4812-b683-ddb1a828b8e1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.375782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.375995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.376697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.377050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.377346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.377475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.378239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.378804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.379902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:50.380308 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.380479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:50.382876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.397881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:50.404119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1215/2432] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:16:50.408715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.422043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4f281-dc0a-4cc7-ad12-a3b1b2ffd9c5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.430905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.431995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.431995 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.432462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.433324 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.435742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.492091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.493556 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.508961 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1218/2433] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:16:50.559003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1216/2434] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:50 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:50.563842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.589292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd987e-068a-4415-9ccd-6007d795542c service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.697799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:50.712356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.716208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.716722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.717023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.717819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.722215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.867244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1219/2435] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:50 2022] POST /identity/v3/users => generated 287 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:50.872099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.886998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816de8a-c25a-42c9-ad1d-4acc21f791c6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.894975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.896988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.909174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.909174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.909174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.909174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.925290 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1217/2436] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:16:50.926292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:50.955320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee317f-6f03-411f-8e0e-2a1068796e05 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:50.972217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9868b-78f6-4f26-81cd-c396cb907383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:50.973343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9868b-78f6-4f26-81cd-c396cb907383 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:50.973726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9868b-78f6-4f26-81cd-c396cb907383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:50.974102 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9868b-78f6-4f26-81cd-c396cb907383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:50.975145 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9868b-78f6-4f26-81cd-c396cb907383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:50.983530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9868b-78f6-4f26-81cd-c396cb907383 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:50.998967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1220/2437] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:51.012666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:51.027147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048b661-1d18-4b9d-97d4-482f39c16756 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:51.032648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:51.033972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:51.034325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:51.034607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] PATH_INFO: `/v3/projects/721f8a391d2c47ae8480762267d81b49/users/7c73a7929c4b4da38622fee31f91686f/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:51.065441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=721f8a391d2c47ae8480762267d81b49, user_id=7c73a7929c4b4da38622fee31f91686f, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:51.069695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:51.265158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1218/2438] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:51 2022] PUT /identity/v3/projects/721f8a391d2c47ae8480762267d81b49/users/7c73a7929c4b4da38622fee31f91686f/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:51.279205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:51.280136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:51.280136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:51.316342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] MFA Rules not processed for user `7c73a7929c4b4da38622fee31f91686f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:51.317791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:51.430759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1221/2439] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:51.436096 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:51.565228 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb7985-462c-4faa-a9fd-2ef865ca45fc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:51.574550 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:51.575320 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:51.576044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:51.576738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:51.578799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:51.583052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:51.690456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1219/2440] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:51 2022] POST /identity/v3/projects => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:51.696989 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:51.725188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282b0c1-4136-42e2-b3ba-48f5a18e9485 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:51.809660 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:51.810917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:51.810917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:51.811346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:51.812197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:51.814419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:51.975713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1222/2441] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:51 2022] POST /identity/v3/users => generated 292 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:16:51.979002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:52.003521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076820f4-750b-4198-ab05-e1942ab9aaf7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:52.036563 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:52.042132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.042716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.043166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.044486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:52.047037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:52.078164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1220/2442] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:52.079881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:52.102191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf993891-77fa-463d-bbf0-c923360464b0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:52.110715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:52.112463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.113058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.113495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] PATH_INFO: `/v3/projects/31287003096941aa9b08b6c960745c67/users/e15ff59efeb148eca7bcd4c3d65bf37c/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.142514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31287003096941aa9b08b6c960745c67, user_id=e15ff59efeb148eca7bcd4c3d65bf37c, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:52.145680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:52.307832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1223/2443] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:52 2022] PUT /identity/v3/projects/31287003096941aa9b08b6c960745c67/users/e15ff59efeb148eca7bcd4c3d65bf37c/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:52.313184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:52.334336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4dc87-5b72-4563-a8a2-9aba55c4ee65 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:52.437915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b28abe-0c21-4e3b-bcb4-2242af993e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:52.439766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b28abe-0c21-4e3b-bcb4-2242af993e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.440599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b28abe-0c21-4e3b-bcb4-2242af993e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.441190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b28abe-0c21-4e3b-bcb4-2242af993e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.442335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b28abe-0c21-4e3b-bcb4-2242af993e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:52.444479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b28abe-0c21-4e3b-bcb4-2242af993e18 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:52.465072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1221/2444] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:16:52.468965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:52.481709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff19c1-7c9f-4d18-b234-28a21944347d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:52.488292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:52.489228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.489570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.489891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] PATH_INFO: `/v3/projects/31287003096941aa9b08b6c960745c67/users/e15ff59efeb148eca7bcd4c3d65bf37c/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.493224 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=31287003096941aa9b08b6c960745c67, user_id=e15ff59efeb148eca7bcd4c3d65bf37c, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:52.496711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:52.522457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1224/2445] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:16:52 2022] PUT /identity/v3/projects/31287003096941aa9b08b6c960745c67/users/e15ff59efeb148eca7bcd4c3d65bf37c/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:52.524949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.525249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.526499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.568910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] MFA Rules not processed for user `e15ff59efeb148eca7bcd4c3d65bf37c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:52.570849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:52.674265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1222/2446] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:52.686272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:52.763487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423699f-7efe-4e56-8971-8a13884dabdc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:52.767918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdb90d-0225-4ec2-bf8d-b93ca9c0c19e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:52.768518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdb90d-0225-4ec2-bf8d-b93ca9c0c19e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:52.768817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdb90d-0225-4ec2-bf8d-b93ca9c0c19e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:52.769115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdb90d-0225-4ec2-bf8d-b93ca9c0c19e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:52.782693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdb90d-0225-4ec2-bf8d-b93ca9c0c19e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:52.784191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdb90d-0225-4ec2-bf8d-b93ca9c0c19e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:52.797016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1225/2447] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:52 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:16:53.413017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:53.812950 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:54.706131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:54.831216 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af26ab8-5df9-4f95-b92f-80a75bc181e4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:54.837512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:54.839982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:54.840609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:54.841099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] PATH_INFO: `/v3/users/7c73a7929c4b4da38622fee31f91686f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:54.843742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c73a7929c4b4da38622fee31f91686f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:54.847496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:55.171993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c73a7929c4b4da38622fee31f91686f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:55.172716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c73a7929c4b4da38622fee31f91686f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:55.291777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c73a7929c4b4da38622fee31f91686f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:55.300697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c73a7929c4b4da38622fee31f91686f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:55.321593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c73a7929c4b4da38622fee31f91686f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:55.329586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1223/2448] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:54 2022] DELETE /identity/v3/users/7c73a7929c4b4da38622fee31f91686f => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:55.334798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8543-9388-4432-8c79-6eb6c5f90dc3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:55.335454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8543-9388-4432-8c79-6eb6c5f90dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:55.335647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d8543-9388-4432-8c79-6eb6c5f90dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:55.354319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ca5d8543-9388-4432-8c79-6eb6c5f90dc3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:55.355585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca5d8543-9388-4432-8c79-6eb6c5f90dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:55.421039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1226/2449] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:55.435515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:55.437293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:55.518453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a2d0d-5b06-42d3-b6de-f09ed0029492 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:55.522617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:55.523462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:55.523881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:55.524433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:55.538169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:55.539809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:55.550007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1224/2450] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:16:55.732410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:56.045170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:56.045759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:56.046200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:56.069631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:56.071111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:56.138267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1227/2451] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jun 16 18:16:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:16:56.167946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:56.170391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:56.229610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:56.231332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:56.253342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1c741-716e-42a9-ba8b-cbcbb8ab9362 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:56.269420 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:56.270662 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:56.273790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:56.274435 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] PATH_INFO: `/v3/projects/721f8a391d2c47ae8480762267d81b49` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:56.276092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=721f8a391d2c47ae8480762267d81b49)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:56.278796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:56.291482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:56.313347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ffeac-be7f-4afb-97fc-494637512630 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:56.322084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:56.322881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:56.323274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:56.323683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:56.338489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:56.343218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:56.359636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1225/2452] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:16:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:16:56.364600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:56.367092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:56.394359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d78aa-a82e-4564-8ace-b25448026c06 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:56.401292 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:56.402325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:56.402661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:56.402843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] PATH_INFO: `/v3/users/5d1510e90ab84fa58cb9fd93583490fe` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:56.404690 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1510e90ab84fa58cb9fd93583490fe)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:56.406429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:56.498731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721f8a391d2c47ae8480762267d81b49', 'request_id': 'req-d71111dd-62f5-456d-ba78-1bb8ba2fc921'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.761106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721f8a391d2c47ae8480762267d81b49', 'request_id': 'req-d71111dd-62f5-456d-ba78-1bb8ba2fc921'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.774742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721f8a391d2c47ae8480762267d81b49', 'request_id': 'req-d71111dd-62f5-456d-ba78-1bb8ba2fc921'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.795404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] The token cache is being invalidate because project 721f8a391d2c47ae8480762267d81b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:56.795899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.797397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.800589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1228/2453] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:56 2022] DELETE /identity/v3/projects/721f8a391d2c47ae8480762267d81b49 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:16:56.807498 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:56.810087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:56.878771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1510e90ab84fa58cb9fd93583490fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.881326 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1510e90ab84fa58cb9fd93583490fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:56.945113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71111dd-62f5-456d-ba78-1bb8ba2fc921 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:56.955439 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:56.956867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:56.957292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:56.957732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] PATH_INFO: `/v3/users/e15ff59efeb148eca7bcd4c3d65bf37c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:56.961799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15ff59efeb148eca7bcd4c3d65bf37c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:56.965595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:57.019509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1510e90ab84fa58cb9fd93583490fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.031055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1510e90ab84fa58cb9fd93583490fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.048973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-71adfbd7-b686-449b-8f8a-63520f4badb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1510e90ab84fa58cb9fd93583490fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.057354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1226/2454] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:56 2022] DELETE /identity/v3/users/5d1510e90ab84fa58cb9fd93583490fe => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:57.064710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:57.065276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:57.065818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:57.090642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:57.092126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:57.195943 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1227/2455] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:57.217437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:57.217437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:57.283947 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15ff59efeb148eca7bcd4c3d65bf37c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.306673 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15ff59efeb148eca7bcd4c3d65bf37c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.314169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42885e-98d1-4d84-bc78-dceb71d508b3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:57.321153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:57.321557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15ff59efeb148eca7bcd4c3d65bf37c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.322131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:57.322495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:57.322841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:57.338684 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:57.340678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:57.363982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1228/2456] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:16:57.606519 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15ff59efeb148eca7bcd4c3d65bf37c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.607183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15ff59efeb148eca7bcd4c3d65bf37c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:57.613786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1229/2457] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:56 2022] DELETE /identity/v3/users/e15ff59efeb148eca7bcd4c3d65bf37c => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:16:58.896959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:58.898646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:58.914485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7d0be-1e4b-4627-ad76-606760af68a4 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:58.929624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:58.931829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:59.002082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:59.002971 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.003258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.005237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] PATH_INFO: `/v3/projects/31287003096941aa9b08b6c960745c67` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.005237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=31287003096941aa9b08b6c960745c67)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:59.012287 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:59.060718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30d650-e1e0-435e-89ab-f2523b81c881 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:59.066852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:59.067786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.068622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.068904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] PATH_INFO: `/v3/projects/e1a929467e8641eda2fcbb95cd157e57` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.070214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a929467e8641eda2fcbb95cd157e57)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:59.072180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:59.187773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31287003096941aa9b08b6c960745c67', 'request_id': 'req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.195210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31287003096941aa9b08b6c960745c67', 'request_id': 'req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.202016 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31287003096941aa9b08b6c960745c67', 'request_id': 'req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.210323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a929467e8641eda2fcbb95cd157e57', 'request_id': 'req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.272240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] The token cache is being invalidate because project 31287003096941aa9b08b6c960745c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:59.272592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.275329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b7c6a2e-1a60-46c8-9d28-61356425dd1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.281053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1229/2458] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:58 2022] DELETE /identity/v3/projects/31287003096941aa9b08b6c960745c67 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:16:59.283107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700273b-cdd9-4c15-9e38-22a76ea5041b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.283638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700273b-cdd9-4c15-9e38-22a76ea5041b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.283929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700273b-cdd9-4c15-9e38-22a76ea5041b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.306519 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1700273b-cdd9-4c15-9e38-22a76ea5041b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:59.307732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1700273b-cdd9-4c15-9e38-22a76ea5041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:59.378112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a929467e8641eda2fcbb95cd157e57', 'request_id': 'req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.388646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a929467e8641eda2fcbb95cd157e57', 'request_id': 'req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.404032 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1230/2459] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:59.406578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] The token cache is being invalidate because project e1a929467e8641eda2fcbb95cd157e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:16:59.406975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.407800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.408624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.409093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.409899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.410098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1230/2460] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:16:58 2022] DELETE /identity/v3/projects/e1a929467e8641eda2fcbb95cd157e57 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:16:59.415477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:59.417106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:59.450624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c81e4-24f4-4c86-bc82-b4cdda83f6fa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:59.450818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:16:59.450818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:16:59.451019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:59.451019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.451019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.451019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] PATH_INFO: `/v3/users/b44bd9877a5048af939404b75ec85c92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.451461 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44bd9877a5048af939404b75ec85c92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:59.454045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:59.465803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1231/2461] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:16:59.470025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:59.483803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fcfd7-b9e2-473b-9f05-3757f1f2b21d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:59.490761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:59.490761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.490761 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.491177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.491992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:16:59.492303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:59.493965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:59.509778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1232/2462] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:16:59.514375 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:59.527723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e8e7f-303b-4890-ac37-1ac1cc60dc8d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:59.533124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:59.537638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.537638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.537638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.537638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:59.538839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:59.704015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:16:59.712770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1233/2463] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:16:59 2022] POST /identity/v3/projects => generated 363 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:59.719060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:16:59.808123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec81f5-a166-4d85-9111-fd25d2e68b22 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:16:59.819480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:16:59.820198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:16:59.825430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:16:59.825430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:16:59.825430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:16:59.825430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:16:59.833668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44bd9877a5048af939404b75ec85c92'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.854674 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44bd9877a5048af939404b75ec85c92'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.864412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44bd9877a5048af939404b75ec85c92'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.977678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44bd9877a5048af939404b75ec85c92'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.978648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44bd9877a5048af939404b75ec85c92'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:16:59.986810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1234/2464] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:16:59 2022] POST /identity/v3/users => generated 282 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:16:59.987415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1231/2465] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:16:59 2022] DELETE /identity/v3/users/b44bd9877a5048af939404b75ec85c92 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:00.002309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:00.004209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:00.018845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cbd02-7319-41f9-9752-51e6e787738a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:00.094786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:00.095823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.096115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.096392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.097139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:00.098742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:00.118305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1235/2466] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:00.122709 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:00.134476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698241e5-73f7-42e2-b081-764815e88874 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:00.141264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7aebc-0275-4866-872a-d620413e5d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:00.141264 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7aebc-0275-4866-872a-d620413e5d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.141677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7aebc-0275-4866-872a-d620413e5d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.141677 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7aebc-0275-4866-872a-d620413e5d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.142516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7aebc-0275-4866-872a-d620413e5d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:00.144333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7aebc-0275-4866-872a-d620413e5d3e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:00.156630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1232/2467] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:00.162281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:00.175104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857e209-5455-41c7-b4a5-391f3ed94b61 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:00.181075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:00.181996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.182281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.182562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] PATH_INFO: `/v3/projects/e1a3b67eed194357aae7175c3c051703/users/2739adae48c140da966e1c004da3515b/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.199755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a3b67eed194357aae7175c3c051703, user_id=2739adae48c140da966e1c004da3515b, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:00.203145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:00.207471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:00.335143 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1236/2468] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:00 2022] PUT /identity/v3/projects/e1a3b67eed194357aae7175c3c051703/users/2739adae48c140da966e1c004da3515b/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:00.358885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.359177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.359616 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.396157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] MFA Rules not processed for user `2739adae48c140da966e1c004da3515b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:00.397977 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:00.497800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1233/2469] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:00.503175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:00.597023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5543a-9b89-4dab-a17c-aa8938d036a6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:00.603140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:00.603842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.604122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.604394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.605116 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:00.607031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:00.707125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:00.709220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:00.736955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d817-9c84-48ae-a27b-9b59d4d3f14b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:00.753806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:00.753806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.753806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.753806 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] PATH_INFO: `/v3/projects/44c9ff8b45d14e9fb51eeb297f55d821` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.756136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c9ff8b45d14e9fb51eeb297f55d821)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:00.756486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1237/2470] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:00 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:00.756588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:00.757361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:00.858902 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2feadfc-5823-4dfc-afa5-057ac30fc1ff admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:00.863916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:00.864594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:00.865028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:00.865779 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:00.866095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:00.867936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:00.898136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c9ff8b45d14e9fb51eeb297f55d821', 'request_id': 'req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.092702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1238/2471] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:00 2022] POST /identity/v3/users => generated 289 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:01.097339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.109721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5386-6a74-40ec-a3db-16e3f73623bc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.154759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c9ff8b45d14e9fb51eeb297f55d821', 'request_id': 'req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.164180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c9ff8b45d14e9fb51eeb297f55d821', 'request_id': 'req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.179411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] The token cache is being invalidate because project 44c9ff8b45d14e9fb51eeb297f55d821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:01.179411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.180168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.181751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1234/2472] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:00 2022] DELETE /identity/v3/projects/44c9ff8b45d14e9fb51eeb297f55d821 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:01.186492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.186703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.187186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.187818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.188139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.188904 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:01.189049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.191154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.205940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5b26e-4c11-45d3-8b15-c0c4d6c9e8e1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.207564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1239/2473] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:01.211464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.213133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:01.213605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.214506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.214788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.215063 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] PATH_INFO: `/v3/users/fbef57c892ce451d9b950b4d2f20d51c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.216313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbef57c892ce451d9b950b4d2f20d51c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.218790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.226665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be9e14-864e-45b7-8a58-81307ab75fc7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.233195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.234189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.234480 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.234764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] PATH_INFO: `/v3/projects/a170e71701714df78642532067f70e0e/users/e46549533bd24bc6b73e82708600ea66/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.254965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a170e71701714df78642532067f70e0e, user_id=e46549533bd24bc6b73e82708600ea66, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.259120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.504277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1240/2474] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:01 2022] PUT /identity/v3/projects/a170e71701714df78642532067f70e0e/users/e46549533bd24bc6b73e82708600ea66/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:01.509440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.562735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbef57c892ce451d9b950b4d2f20d51c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.584046 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbef57c892ce451d9b950b4d2f20d51c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.602676 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbef57c892ce451d9b950b4d2f20d51c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.610628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36e30-91cf-4a8d-ad66-6ae90c037b24 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.620398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.621568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.622054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.622515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.630307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.630307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.647155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1241/2475] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:01.652060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.666151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4df70-ca67-4a4e-86f0-ae31b0773eeb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.674195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b3eb3-c0d4-4116-8df6-3d822bb833b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.681610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b3eb3-c0d4-4116-8df6-3d822bb833b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.681610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b3eb3-c0d4-4116-8df6-3d822bb833b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.681610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b3eb3-c0d4-4116-8df6-3d822bb833b1 admin admin] PATH_INFO: `/v3/projects/a170e71701714df78642532067f70e0e/users/e46549533bd24bc6b73e82708600ea66/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.681610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b3eb3-c0d4-4116-8df6-3d822bb833b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a170e71701714df78642532067f70e0e, user_id=e46549533bd24bc6b73e82708600ea66, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.684632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b3eb3-c0d4-4116-8df6-3d822bb833b1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:01.701702 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1242/2476] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:01 2022] PUT /identity/v3/projects/a170e71701714df78642532067f70e0e/users/e46549533bd24bc6b73e82708600ea66/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:01.710212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbef57c892ce451d9b950b4d2f20d51c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.716871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbef57c892ce451d9b950b4d2f20d51c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:01.717037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.717037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.717037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.722761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1235/2477] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:01 2022] DELETE /identity/v3/users/fbef57c892ce451d9b950b4d2f20d51c => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:01.753107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] MFA Rules not processed for user `e46549533bd24bc6b73e82708600ea66`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:01.754485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:01.863776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1243/2478] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:01.890535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:01.892428 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:01.972183 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9e489-cb7f-4b13-b006-26cf373dfd71 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:01.977237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:01.986101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:01.986101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:01.986101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:01.995240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:01.997686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:02.017023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1236/2479] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:02.217446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:02.414961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:02.416642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:02.440225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:02.459658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f964-cbb0-4ee1-b50f-f6c9ecde4542 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:02.463740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:02.464398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:02.464681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:02.464970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:02.467125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:02.520733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9f65f-10b3-4f46-b969-080fbb7bda23 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:02.526566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:02.527474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:02.527760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:02.528306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] PATH_INFO: `/v3/projects/843dcb5449ab4f8dba246b9377d939dd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:02.529982 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=843dcb5449ab4f8dba246b9377d939dd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:02.532021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:02.555991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:02.557810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d636f2a-8d32-47fd-a960-04a5239041cb service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:02.569985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:02.571933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1237/2480] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:02.716157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843dcb5449ab4f8dba246b9377d939dd', 'request_id': 'req-4e47458f-215a-49eb-8fe2-318fdb0b3748'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:02.727905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843dcb5449ab4f8dba246b9377d939dd', 'request_id': 'req-4e47458f-215a-49eb-8fe2-318fdb0b3748'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:02.737135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843dcb5449ab4f8dba246b9377d939dd', 'request_id': 'req-4e47458f-215a-49eb-8fe2-318fdb0b3748'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:02.897991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] The token cache is being invalidate because project 843dcb5449ab4f8dba246b9377d939dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:02.908231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:02.908231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:02.908231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1244/2481] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:02 2022] DELETE /identity/v3/projects/843dcb5449ab4f8dba246b9377d939dd => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:03.898390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:04.092404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:04.761330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:05.547673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:15.040734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=58,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:15.989311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:23.096117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81866bea-d276-438f-97cd-70a5ad630d73 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:23.097012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81866bea-d276-438f-97cd-70a5ad630d73 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:23.097231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81866bea-d276-438f-97cd-70a5ad630d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:23.125065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-81866bea-d276-438f-97cd-70a5ad630d73 None None] MFA Rules not processed for user `f8cc86b384df4223a800838a3d386d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:23.131477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-81866bea-d276-438f-97cd-70a5ad630d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:23.200097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1238/2482] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:25.228462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:25.230619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:25.340298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47458f-215a-49eb-8fe2-318fdb0b3748 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:25.348782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:25.350765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:25.351126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:25.351498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] PATH_INFO: `/v3/users/2739adae48c140da966e1c004da3515b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:25.353131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2739adae48c140da966e1c004da3515b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:25.355582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:25.929845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2739adae48c140da966e1c004da3515b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:25.930526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2739adae48c140da966e1c004da3515b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:26.053617 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2739adae48c140da966e1c004da3515b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:26.063892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2739adae48c140da966e1c004da3515b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:26.077596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2739adae48c140da966e1c004da3515b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:26.084415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1245/2483] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:25 2022] DELETE /identity/v3/users/2739adae48c140da966e1c004da3515b => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:26.088923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:26.089585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:26.089585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:26.111006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:26.112294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:26.182967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1239/2484] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:26.193786 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:26.195541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:26.267109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde135ef-3e08-4c5e-8319-df7d004ece80 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:26.271302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:26.272063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:26.272410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:26.272741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:26.284954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:26.288394 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:26.304184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1246/2485] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:26.913902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:26.916121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:26.940532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44410031-9e79-49a6-9e9a-35a57f51f983 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:26.951726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:26.951726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:26.951726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:26.951726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] PATH_INFO: `/v3/projects/e1a3b67eed194357aae7175c3c051703` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:26.955079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a3b67eed194357aae7175c3c051703)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:26.958220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:27.135239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a3b67eed194357aae7175c3c051703', 'request_id': 'req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.252066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a3b67eed194357aae7175c3c051703', 'request_id': 'req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.262834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a3b67eed194357aae7175c3c051703', 'request_id': 'req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.275015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] The token cache is being invalidate because project e1a3b67eed194357aae7175c3c051703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:27.282539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.282539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.282539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1240/2486] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:26 2022] DELETE /identity/v3/projects/e1a3b67eed194357aae7175c3c051703 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:27.283001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:27.292796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43718fc7-b418-4768-9c8a-6af01cc6033d service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:27.313654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:27.355432 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:27.356329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:27.356615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:27.356893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] PATH_INFO: `/v3/users/e46549533bd24bc6b73e82708600ea66` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:27.358100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46549533bd24bc6b73e82708600ea66)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:27.360110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:27.684105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46549533bd24bc6b73e82708600ea66'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.684790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46549533bd24bc6b73e82708600ea66'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.800742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46549533bd24bc6b73e82708600ea66'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.812126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46549533bd24bc6b73e82708600ea66'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.830952 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b481b8cf-1e18-40b4-919e-dabe1881343d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46549533bd24bc6b73e82708600ea66'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:27.841059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1247/2487] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:27 2022] DELETE /identity/v3/users/e46549533bd24bc6b73e82708600ea66 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:28.494996 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:28.497039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:28.512386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08ad1e-0bab-43b2-b5c4-f9c371a778bb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:28.546348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:28.580765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:28.581650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:28.581938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:28.582213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] PATH_INFO: `/v3/projects/a170e71701714df78642532067f70e0e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:28.584025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a170e71701714df78642532067f70e0e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:28.585982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:28.860167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a170e71701714df78642532067f70e0e', 'request_id': 'req-aaa7db55-e575-498c-b339-65132cf665a2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:28.967475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a170e71701714df78642532067f70e0e', 'request_id': 'req-aaa7db55-e575-498c-b339-65132cf665a2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:28.977365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a170e71701714df78642532067f70e0e', 'request_id': 'req-aaa7db55-e575-498c-b339-65132cf665a2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:28.990234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] The token cache is being invalidate because project a170e71701714df78642532067f70e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:28.990796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:28.991665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaa7db55-e575-498c-b339-65132cf665a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:28.998827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:28.999310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:28.999510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.002201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1241/2488] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:28 2022] DELETE /identity/v3/projects/a170e71701714df78642532067f70e0e => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:29.022697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:29.069117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:29.122697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1248/2489] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:29.133135 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:29.133458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:29.133781 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.154287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:29.155584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:29.165160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1242/2490] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:29.169524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:29.180838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c655ce-b435-45e1-853a-0112305a4bb5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:29.187047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:29.188033 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:29.188283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:29.188555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.189296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:29.189607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:29.191222 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:29.213496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1249/2491] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:29.220826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:29.228975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf26b7-7f84-4186-8755-0c19392a541c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:29.241553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:29.248889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:29.259216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:29.259721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.261232 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:29.263306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:29.563335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1243/2492] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:29 2022] POST /identity/v3/projects => generated 363 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:29.568347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:29.580025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dc927-a54e-40c9-a0c9-b7e72dd2f7c2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:29.614428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:29.650753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:29.651500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:29.651789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:29.652077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.652789 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:29.654718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:29.849391 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1250/2493] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:29 2022] POST /identity/v3/users => generated 282 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:29.861501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:29.875873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40bd44-f1e7-42a6-9938-5dafa4750844 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:29.881179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:29.882130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:29.882415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:29.882700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.883477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:29.885384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:29.907951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1244/2494] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:29.908957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:29.925057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529d1b-648a-424b-9ee3-50f74f63e861 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:29.934183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221da4b-22ac-46ec-a847-0ee66f8c7464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:29.935171 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221da4b-22ac-46ec-a847-0ee66f8c7464 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:29.936913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221da4b-22ac-46ec-a847-0ee66f8c7464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:29.937592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221da4b-22ac-46ec-a847-0ee66f8c7464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:29.938917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221da4b-22ac-46ec-a847-0ee66f8c7464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:29.942063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221da4b-22ac-46ec-a847-0ee66f8c7464 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:29.969838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1251/2495] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:29.973027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:29.990160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10826122-48b5-425d-80cb-640ea12a4ab4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:30.000432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:30.000432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:30.000432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:30.000432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] PATH_INFO: `/v3/projects/b1052763030d456cb741bf6545b4aa07/users/4fbfab81adbe40ee93c2a429a139b784/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:30.034347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1052763030d456cb741bf6545b4aa07, user_id=4fbfab81adbe40ee93c2a429a139b784, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:30.038141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:30.283458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1245/2496] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:29 2022] PUT /identity/v3/projects/b1052763030d456cb741bf6545b4aa07/users/4fbfab81adbe40ee93c2a429a139b784/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:30.289447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:30.290000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:30.290232 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:30.303815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:30.305881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:30.332829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] MFA Rules not processed for user `4fbfab81adbe40ee93c2a429a139b784`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:30.335044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:30.408064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cf0a9-0716-4abc-8471-a9809bdfd547 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:30.413948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:30.414965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:30.415641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:30.415922 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:30.419257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:30.432570 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1252/2497] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:30.432805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:30.433064 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:30.433813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:30.436900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:30.446959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1246/2498] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:30.544689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb1e916-0c12-4b18-b7f0-ff2a24e48fcd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:30.551846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:30.554145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:30.555416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:30.555845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:30.557126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:30.560660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:30.733129 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1253/2499] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:30 2022] POST /identity/v3/projects => generated 365 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:30.734279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:30.747024 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270c282-a324-4988-b58a-f7b1c9e50cd6 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:30.838323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:30.839019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:30.839325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:30.839617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:30.840339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:30.842363 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:30.963984 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1247/2500] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:30 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:30.968333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:30.982518 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b7e92-881c-4075-a5af-5707d27314d0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:30.988066 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:30.989107 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:30.989395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:30.989748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:30.990541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:30.992300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:31.006452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1254/2501] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:31.016528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:31.033932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a020e1d-c071-461c-a503-e7e8301a885e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:31.042602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:31.043716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:31.044378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:31.046368 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] PATH_INFO: `/v3/projects/c19de8b489f94cfc90491c1323c79458/users/1f00f6d1fcaf4813a869d8775c38e88a/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:31.077438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19de8b489f94cfc90491c1323c79458, user_id=1f00f6d1fcaf4813a869d8775c38e88a, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:31.080573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:31.163640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1248/2502] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:31 2022] PUT /identity/v3/projects/c19de8b489f94cfc90491c1323c79458/users/1f00f6d1fcaf4813a869d8775c38e88a/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:31.168469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:31.183794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9d5a2-113e-4f7d-85ff-e4f30c19cdc3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:31.314780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5d2f4-5637-412c-ae4d-62f975257aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:31.315885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d2f4-5637-412c-ae4d-62f975257aea admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:31.316251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d2f4-5637-412c-ae4d-62f975257aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:31.316654 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d2f4-5637-412c-ae4d-62f975257aea admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:31.317502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5d2f4-5637-412c-ae4d-62f975257aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:31.320506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5d2f4-5637-412c-ae4d-62f975257aea admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:31.332597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1255/2503] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:31.338094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:31.352530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f4bf0-69c7-4da1-b8dd-5c0632eff312 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:31.358227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:31.360017 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:31.360516 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:31.361204 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] PATH_INFO: `/v3/projects/c19de8b489f94cfc90491c1323c79458/users/1f00f6d1fcaf4813a869d8775c38e88a/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:31.365908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19de8b489f94cfc90491c1323c79458, user_id=1f00f6d1fcaf4813a869d8775c38e88a, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:31.371265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:31.390614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1249/2504] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:31 2022] PUT /identity/v3/projects/c19de8b489f94cfc90491c1323c79458/users/1f00f6d1fcaf4813a869d8775c38e88a/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:31.406043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:31.406514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:31.407123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:31.449506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:31.452902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:31.486717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] MFA Rules not processed for user `1f00f6d1fcaf4813a869d8775c38e88a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:31.488987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:31.627601 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc50a5-be09-4cd0-a6de-df1fced23856 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:31.639584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:31.640504 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1256/2505] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:31.640790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:31.641576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:31.642018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:31.646899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:31.653698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:31.655198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:31.674251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:31.683398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:31.697041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1250/2506] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:31.752204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8334177-284f-4b7e-b1c7-678668caaa90 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:31.758430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:31.767078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:31.767078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:31.767078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:31.777167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:31.778782 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:31.790716 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1257/2507] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:31.849638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:32.152898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:32.163205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673795c-e415-415f-98e5-953350fc3822 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:32.166870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:32.167555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:32.167845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:32.168124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:32.245604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:32.250954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:32.267901 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1251/2508] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:32.795700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:35.685686 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:35.687190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:35.732334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:35.752403 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edc890-1374-481d-b991-df34b4d273f8 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:35.757025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:35.760705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:35.760705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:35.760705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:35.760705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:35.872250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:35.873778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d591f7-7084-4d76-b2b1-454aa572629d service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:35.894708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1258/2509] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:36.867160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:36.939298 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582de5-04d9-4230-b459-1dfe16ffe6b7 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:36.945808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:36.945808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:36.946121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:36.946443 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] PATH_INFO: `/v3/users/4fbfab81adbe40ee93c2a429a139b784` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:36.947710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbfab81adbe40ee93c2a429a139b784)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:36.950287 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:37.287863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbfab81adbe40ee93c2a429a139b784'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:37.309826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbfab81adbe40ee93c2a429a139b784'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:37.318352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbfab81adbe40ee93c2a429a139b784'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:37.425100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbfab81adbe40ee93c2a429a139b784'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:37.427816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbfab81adbe40ee93c2a429a139b784'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:37.441032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1252/2510] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:36 2022] DELETE /identity/v3/users/4fbfab81adbe40ee93c2a429a139b784 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:37.446923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:37.447514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:37.447718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:37.470217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:37.471493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:37.558854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1259/2511] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:37.572721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:37.594099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:37.656863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19433dc0-3d91-4c5b-8623-1861645a0168 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:37.673927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:37.673927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:37.673927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:37.673927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:37.674751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:37.675732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:37.679671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:37.694580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1253/2512] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:38.351069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:38.353283 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:38.388711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ae3e3-5583-4401-a7ba-c002056fa187 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:38.394731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:38.396735 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:38.402674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:38.402674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] PATH_INFO: `/v3/projects/b1052763030d456cb741bf6545b4aa07` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:38.402674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1052763030d456cb741bf6545b4aa07)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:38.404700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:38.596037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1052763030d456cb741bf6545b4aa07', 'request_id': 'req-681e8c47-d3da-4705-b7ce-5094cf07f36a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:38.602406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:38.603939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1052763030d456cb741bf6545b4aa07', 'request_id': 'req-681e8c47-d3da-4705-b7ce-5094cf07f36a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:38.610628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1052763030d456cb741bf6545b4aa07', 'request_id': 'req-681e8c47-d3da-4705-b7ce-5094cf07f36a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:38.789742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] The token cache is being invalidate because project b1052763030d456cb741bf6545b4aa07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:38.789963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:38.790603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:38.792195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1260/2513] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:38 2022] DELETE /identity/v3/projects/b1052763030d456cb741bf6545b4aa07 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:38.796397 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:38.807618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed8735-f09f-4382-8a39-d508554cd1f8 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:38.880710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:38.881622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:38.881907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:38.882284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] PATH_INFO: `/v3/users/1f00f6d1fcaf4813a869d8775c38e88a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:38.883441 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f00f6d1fcaf4813a869d8775c38e88a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:38.885405 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:39.178369 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f00f6d1fcaf4813a869d8775c38e88a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:39.197893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f00f6d1fcaf4813a869d8775c38e88a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:39.208926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f00f6d1fcaf4813a869d8775c38e88a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:39.328002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f00f6d1fcaf4813a869d8775c38e88a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:39.328887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-612087d8-7a05-465c-b37d-565285072369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f00f6d1fcaf4813a869d8775c38e88a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:39.339968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1254/2514] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:38 2022] DELETE /identity/v3/users/1f00f6d1fcaf4813a869d8775c38e88a => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:39.901390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:39.903163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:39.919511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e8c47-d3da-4705-b7ce-5094cf07f36a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:39.996098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:39.997031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:39.997323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:39.997609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] PATH_INFO: `/v3/projects/c19de8b489f94cfc90491c1323c79458` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:39.999006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19de8b489f94cfc90491c1323c79458)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:40.001062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:40.174087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19de8b489f94cfc90491c1323c79458', 'request_id': 'req-edec05a2-7ec0-4694-97f6-b8515f89393a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:40.182886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19de8b489f94cfc90491c1323c79458', 'request_id': 'req-edec05a2-7ec0-4694-97f6-b8515f89393a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:40.190683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19de8b489f94cfc90491c1323c79458', 'request_id': 'req-edec05a2-7ec0-4694-97f6-b8515f89393a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:40.339648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] The token cache is being invalidate because project c19de8b489f94cfc90491c1323c79458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:40.339936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:40.340616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-edec05a2-7ec0-4694-97f6-b8515f89393a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:40.342218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1261/2515] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:39 2022] DELETE /identity/v3/projects/c19de8b489f94cfc90491c1323c79458 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:40.347413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.348012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.348012 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.367690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:40.368949 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:40.437812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1255/2516] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:40.446404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.446692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.446974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.472748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:40.474024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:40.486744 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1262/2517] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:40.492059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:40.502198 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76d756-7dae-43ee-aed6-21c1c1e18e77 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:40.508653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:40.509325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.509641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.509915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.510784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:40.511098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:40.512737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:40.526396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1256/2518] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:40.530887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:40.543880 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b6403-f6d6-4f86-8d06-c08591bb285c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:40.547058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:40.547765 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.548052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.548327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.549108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:40.551052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:40.676796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1263/2519] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:40 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:40.681614 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:40.686924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:40.692432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507d165-edb3-4fb6-82e5-6cd9668448d4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:40.752038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:40.752762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.753045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.753323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.754010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:40.757223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:40.882418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1257/2520] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:40 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:40.886759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:40.897709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0042d-0a7c-4686-9d6e-8500add9a64e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:40.902569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:40.903264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.903562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.903853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.904580 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:40.906189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:40.919040 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1264/2521] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:40.923396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:40.936850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7f186-b359-4b93-bb4a-d82d06c04b23 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:40.944485 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3aeb8-7dc7-45ea-adb4-e534df7c5e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:40.946070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3aeb8-7dc7-45ea-adb4-e534df7c5e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.946327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3aeb8-7dc7-45ea-adb4-e534df7c5e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.946610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3aeb8-7dc7-45ea-adb4-e534df7c5e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:40.947631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3aeb8-7dc7-45ea-adb4-e534df7c5e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:40.949313 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3aeb8-7dc7-45ea-adb4-e534df7c5e64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:40.963530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1258/2522] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:40.969111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:40.979983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12fa5a-735c-4262-922c-6c9673789422 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:40.986094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:40.987704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:40.988023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:40.988423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] PATH_INFO: `/v3/projects/5dbb78a88cc54afd8972e2abb75257d3/users/03f284ad694b42ecab89ab768142f674/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.008740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbb78a88cc54afd8972e2abb75257d3, user_id=03f284ad694b42ecab89ab768142f674, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:41.011743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:41.133095 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1265/2523] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:40 2022] PUT /identity/v3/projects/5dbb78a88cc54afd8972e2abb75257d3/users/03f284ad694b42ecab89ab768142f674/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:41.143615 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.143930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.144194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.173701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] MFA Rules not processed for user `03f284ad694b42ecab89ab768142f674`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:41.174932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:41.243619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1259/2524] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:41.255248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:41.270113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:41.326263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a18c8-c4e9-44e3-a9c5-a02b18090fe8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:41.330957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:41.331662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.331959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.332225 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.332926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:41.335801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:41.466954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1266/2525] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:41 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:41.471343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:41.483729 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5add47b-2207-47e2-ae44-ec3520ef84db None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:41.551032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:41.551645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.552268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.553293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.554138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:41.556126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:41.717170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:41.722665 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1260/2526] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:41 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:41.727097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:41.737608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef12e-be45-4afb-bc45-98a50c8fde2e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:41.742749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:41.743526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.743814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.744092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.744826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:41.746489 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:41.758832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1267/2527] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:41.769765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:41.781799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777cb55-0870-4c6b-b278-5411efd4abab admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:41.787475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd34536-ee5b-431c-82bd-4e376aa60e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:41.788223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34536-ee5b-431c-82bd-4e376aa60e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.788515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34536-ee5b-431c-82bd-4e376aa60e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.788799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd34536-ee5b-431c-82bd-4e376aa60e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.789590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd34536-ee5b-431c-82bd-4e376aa60e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:41.791240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd34536-ee5b-431c-82bd-4e376aa60e12 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:41.802793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1261/2528] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:41.806397 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:41.816718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99716e2-17c6-4967-a619-527bd181c931 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:41.822793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:41.823740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.824024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.824299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] PATH_INFO: `/v3/projects/929008b9ac9e4d2198c7cf0dd047b05a/users/884bc598b8014f74b1eea9cd92073a33/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:41.843689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=929008b9ac9e4d2198c7cf0dd047b05a, user_id=884bc598b8014f74b1eea9cd92073a33, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:41.846442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:41.973621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1268/2529] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:41 2022] PUT /identity/v3/projects/929008b9ac9e4d2198c7cf0dd047b05a/users/884bc598b8014f74b1eea9cd92073a33/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:41.985033 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:41.985325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:41.985844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.017407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] MFA Rules not processed for user `884bc598b8014f74b1eea9cd92073a33`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:42.019510 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:42.114815 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1262/2530] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:42.119119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:42.186870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6f7d7-796e-475e-9249-4213d8faa9df admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:42.192288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:42.198126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.198126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.198126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.198126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:42.198126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:42.359261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1269/2531] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:42 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:42.365332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:42.377774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a66edf-75b2-4cd0-b203-dd85a005d139 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:42.457083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:42.458156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.458282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.458491 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.459294 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:42.462436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:42.594580 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1263/2532] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:42 2022] POST /identity/v3/users => generated 296 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:42.599848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:42.612533 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a5e35-5105-4a61-85b6-c0fca28ecf2c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:42.618060 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:42.618734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.619024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.619492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.620089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:42.621700 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:42.633446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1270/2533] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:42.638362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:42.649803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54b9ad-a8fa-494c-a8a6-4d8f4c60a4a9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:42.654508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:42.655438 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.655746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.656029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] PATH_INFO: `/v3/projects/fc5c41d98c6146dbb44ec38cfe82ea21/users/aeca440e9b524fb8b6226f6142aefd93/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.673617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5c41d98c6146dbb44ec38cfe82ea21, user_id=aeca440e9b524fb8b6226f6142aefd93, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:42.676411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:42.769777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1264/2534] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:42 2022] PUT /identity/v3/projects/fc5c41d98c6146dbb44ec38cfe82ea21/users/aeca440e9b524fb8b6226f6142aefd93/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:42.776172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:42.788409 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a17203-0c35-47ff-82a7-7debfe83622a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:42.823799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:42.851013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f720f-60d9-436c-96f2-fe58264af998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:42.852011 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f720f-60d9-436c-96f2-fe58264af998 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.852120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f720f-60d9-436c-96f2-fe58264af998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.852335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f720f-60d9-436c-96f2-fe58264af998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.853076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f720f-60d9-436c-96f2-fe58264af998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:42.854736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f720f-60d9-436c-96f2-fe58264af998 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:42.865926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1271/2535] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:42.871592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:42.883156 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba50636-4089-4a13-8b11-4715bb86306e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:42.888758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:42.889718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.890014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.890291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] PATH_INFO: `/v3/projects/fc5c41d98c6146dbb44ec38cfe82ea21/users/aeca440e9b524fb8b6226f6142aefd93/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.892889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5c41d98c6146dbb44ec38cfe82ea21, user_id=aeca440e9b524fb8b6226f6142aefd93, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:42.896269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:42.908841 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1265/2536] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:42 2022] PUT /identity/v3/projects/fc5c41d98c6146dbb44ec38cfe82ea21/users/aeca440e9b524fb8b6226f6142aefd93/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:42.920166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:42.920479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:42.920768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:42.950574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] MFA Rules not processed for user `aeca440e9b524fb8b6226f6142aefd93`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:42.954615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:43.046048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1272/2537] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:43.062240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:43.064105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:43.105965 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:43.134523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c519c-7119-4d1e-a41c-bde859ef78e8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:43.138830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:43.139536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:43.139821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:43.140097 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:43.150862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:43.152407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:43.162932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1266/2538] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:43.302067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:43.304172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:43.320783 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42c5e8-0c51-476c-ab74-65140d808c52 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:43.328000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:43.329037 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:43.329523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:43.329990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:43.397467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:43.399036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:43.408653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1273/2539] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:45.133611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:45.135790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:45.203378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de920-12be-41ca-8217-16ff4836ff22 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:45.206982 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:45.207640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:45.207928 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:45.208712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:45.210759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:45.281910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:45.283341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:45.293780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1267/2540] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:46.481752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:46.483315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:46.497755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d968d-008c-418e-aa9f-d994029aa0ed service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:46.502545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:46.503523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:46.503807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:46.504281 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:46.507156 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:46.599784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:46.613395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:46.616400 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57e7ea-e647-4fc6-a7df-12077e9df125 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:46.637473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1274/2541] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:49.719300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:49.735777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea7c89-a853-4a41-87b6-7d60b9bdf2cc service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:49.743829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:49.745297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:49.745594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:49.745878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] PATH_INFO: `/v3/users/03f284ad694b42ecab89ab768142f674` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:49.747233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f284ad694b42ecab89ab768142f674)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:49.750144 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:50.055005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f284ad694b42ecab89ab768142f674'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.074077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f284ad694b42ecab89ab768142f674'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.091717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f284ad694b42ecab89ab768142f674'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.100293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:50.252872 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f284ad694b42ecab89ab768142f674'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.253489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f284ad694b42ecab89ab768142f674'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.275502 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1268/2542] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:49 2022] DELETE /identity/v3/users/03f284ad694b42ecab89ab768142f674 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:50.280597 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:50.281126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:50.281247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:50.305384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:50.307492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:50.386335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1275/2543] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:50.397163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:50.399029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:50.478163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1699dd2-d2d4-41d1-83d1-7e8b4cd359c3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:50.489256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:50.489256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:50.490585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:50.490585 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:50.494502 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:50.496429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:50.519080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:50.521158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:50.526662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdfc31-91e6-4988-a6d0-ebef1a766866 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:50.532834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1269/2544] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:50.534297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:50.537776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:50.538374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:50.538992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] PATH_INFO: `/v3/users/8498989e673d445dbc70aa9df55f391b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:50.540727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8498989e673d445dbc70aa9df55f391b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:50.558941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:50.866706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8498989e673d445dbc70aa9df55f391b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.867447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8498989e673d445dbc70aa9df55f391b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.961645 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8498989e673d445dbc70aa9df55f391b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.976214 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8498989e673d445dbc70aa9df55f391b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:50.998097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8498989e673d445dbc70aa9df55f391b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.010956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1276/2545] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:50 2022] DELETE /identity/v3/users/8498989e673d445dbc70aa9df55f391b => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:51.227108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:51.229205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:51.263896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082e11c-c15c-491b-81ce-d215c156b1ad service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:51.326878 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:51.327926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.328277 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.329375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] PATH_INFO: `/v3/projects/5dbb78a88cc54afd8972e2abb75257d3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.329986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbb78a88cc54afd8972e2abb75257d3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:51.332220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:51.387167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:51.388693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:51.410293 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cf89c-9094-48c1-87ba-d2c3e8653394 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:51.418478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:51.419681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.419928 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.420279 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] PATH_INFO: `/v3/projects/5ad7a8c562934b3c843b10d79f31fee5` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.421593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad7a8c562934b3c843b10d79f31fee5)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:51.423636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:51.511087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbb78a88cc54afd8972e2abb75257d3', 'request_id': 'req-1c9f1250-39b3-4925-9e32-a813174342d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.589135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad7a8c562934b3c843b10d79f31fee5', 'request_id': 'req-f22edf85-702d-4a04-b5ed-ef2fcf430f04'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.594310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:51.595741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad7a8c562934b3c843b10d79f31fee5', 'request_id': 'req-f22edf85-702d-4a04-b5ed-ef2fcf430f04'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.602653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad7a8c562934b3c843b10d79f31fee5', 'request_id': 'req-f22edf85-702d-4a04-b5ed-ef2fcf430f04'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.618000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbb78a88cc54afd8972e2abb75257d3', 'request_id': 'req-1c9f1250-39b3-4925-9e32-a813174342d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.624980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbb78a88cc54afd8972e2abb75257d3', 'request_id': 'req-1c9f1250-39b3-4925-9e32-a813174342d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.636612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] The token cache is being invalidate because project 5dbb78a88cc54afd8972e2abb75257d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:51.636899 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.637537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.639084 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1270/2546] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:51 2022] DELETE /identity/v3/projects/5dbb78a88cc54afd8972e2abb75257d3 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:51.642993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:51.644291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:51.713993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f1250-39b3-4925-9e32-a813174342d5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:51.770649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:51.771747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.772031 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.772307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] PATH_INFO: `/v3/users/884bc598b8014f74b1eea9cd92073a33` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.773524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884bc598b8014f74b1eea9cd92073a33)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:51.775528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:51.818778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] The token cache is being invalidate because project 5ad7a8c562934b3c843b10d79f31fee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:51.819073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.819805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f22edf85-702d-4a04-b5ed-ef2fcf430f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:51.821295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1277/2547] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:51 2022] DELETE /identity/v3/projects/5ad7a8c562934b3c843b10d79f31fee5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:51.826091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448b8fc-3601-43e9-aeae-0c554d8fcec0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.826637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448b8fc-3601-43e9-aeae-0c554d8fcec0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.826921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448b8fc-3601-43e9-aeae-0c554d8fcec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.844787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6448b8fc-3601-43e9-aeae-0c554d8fcec0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:51.846012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6448b8fc-3601-43e9-aeae-0c554d8fcec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:51.854774 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1278/2548] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:51.862729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.863017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.863291 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.883135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:51.884398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:51.892260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1279/2549] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:51.896793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:51.905814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d817c-a983-49cd-9c63-95e48d44ad34 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:51.910689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:51.911512 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.911916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.912201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.912926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:51.913234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:51.914843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:51.927208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1280/2550] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:51.932017 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:51.941402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc108cf-7620-475d-b139-fda7523a3322 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:51.946697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:51.947336 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:51.947650 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:51.947949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:51.948637 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:51.950743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.049332 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884bc598b8014f74b1eea9cd92073a33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:52.062579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884bc598b8014f74b1eea9cd92073a33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:52.075342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884bc598b8014f74b1eea9cd92073a33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:52.082465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1281/2551] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:51 2022] POST /identity/v3/projects => generated 405 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:52.083633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:52.146481 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c1465-2a42-46f9-aacb-b6a3f3655bc2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:52.151250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:52.151951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.152274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.152565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.153289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:52.155263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.194992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884bc598b8014f74b1eea9cd92073a33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:52.195639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884bc598b8014f74b1eea9cd92073a33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:52.202436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1271/2552] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:51 2022] DELETE /identity/v3/users/884bc598b8014f74b1eea9cd92073a33 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:52.308262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1282/2553] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:52 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:52.310244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:52.311721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:52.379068 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa957a9-002d-4c11-b884-7f2f6c9f0ab6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:52.384647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:52.385322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.385606 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.385881 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.386688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:52.388256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.402356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1272/2554] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:52.406775 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:52.414890 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955eb2f-dc33-453d-a01f-638626694ae7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:52.419611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2f14b-91f9-4fea-8955-741d692c538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:52.420303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2f14b-91f9-4fea-8955-741d692c538a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.420590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2f14b-91f9-4fea-8955-741d692c538a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.420876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2f14b-91f9-4fea-8955-741d692c538a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.421640 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2f14b-91f9-4fea-8955-741d692c538a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:52.423356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2f14b-91f9-4fea-8955-741d692c538a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.434850 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1283/2555] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:52.441801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:52.454421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2914e00-992d-4b96-895e-9e16339ce2a3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:52.460471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:52.461402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.461688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.461968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] PATH_INFO: `/v3/projects/7adc9fbf58d74cf3a623dd8dacf1a6b8/users/8f7beadcc9954c8287bbc33928a589a6/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.483668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:52.483668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7adc9fbf58d74cf3a623dd8dacf1a6b8, user_id=8f7beadcc9954c8287bbc33928a589a6, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:52.484708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.613407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1273/2556] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:52 2022] PUT /identity/v3/projects/7adc9fbf58d74cf3a623dd8dacf1a6b8/users/8f7beadcc9954c8287bbc33928a589a6/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:52.624182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.624478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.625022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.651948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] MFA Rules not processed for user `8f7beadcc9954c8287bbc33928a589a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:52.653217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:52.718752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1284/2557] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:52.724177 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:52.737938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:52.740455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:52.813058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804ef64-ef87-4a4a-803f-0de7ad2d7fa0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:52.813439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372095b5-64af-4fdb-aa3b-9e8015882610 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:52.819269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:52.819974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.820133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:52.820251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.820527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.821028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:52.821246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:52.821482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:52.821792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] PATH_INFO: `/v3/projects/929008b9ac9e4d2198c7cf0dd047b05a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:52.823076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=929008b9ac9e4d2198c7cf0dd047b05a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:52.823256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.825113 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:52.981360 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1274/2558] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:52 2022] POST /identity/v3/projects => generated 405 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:52.985847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:53.010200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '929008b9ac9e4d2198c7cf0dd047b05a', 'request_id': 'req-1b000f02-eac6-4157-b08e-c652e7ca3711'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.018310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '929008b9ac9e4d2198c7cf0dd047b05a', 'request_id': 'req-1b000f02-eac6-4157-b08e-c652e7ca3711'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.028636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '929008b9ac9e4d2198c7cf0dd047b05a', 'request_id': 'req-1b000f02-eac6-4157-b08e-c652e7ca3711'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.072930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8a936-7e90-4e9d-93af-fc26704e90a9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:53.110543 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] The token cache is being invalidate because project 929008b9ac9e4d2198c7cf0dd047b05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:53.110828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.111538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.113196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1285/2559] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:52 2022] DELETE /identity/v3/projects/929008b9ac9e4d2198c7cf0dd047b05a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:53.118197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:53.119490 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:53.133970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:53.134712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:53.135002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:53.135282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:53.136109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:53.138112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:53.192166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b000f02-eac6-4157-b08e-c652e7ca3711 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:53.197087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:53.198274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:53.198274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:53.198522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] PATH_INFO: `/v3/users/aeca440e9b524fb8b6226f6142aefd93` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:53.199756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeca440e9b524fb8b6226f6142aefd93)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:53.201924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:53.338007 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:53.344237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1275/2560] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:52 2022] POST /identity/v3/users => generated 309 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:53.348160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:53.359782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:53.362386 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38e76-a980-4d2f-8879-d0c82f759afc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:53.367138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:53.367830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:53.368117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:53.368402 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:53.369114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:53.370703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:53.381045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1276/2561] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:53.385092 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:53.394880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbd567-06cc-45fe-9cf5-f8ce7cd2cc9d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:53.401760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:53.402666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:53.417103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:53.417103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] PATH_INFO: `/v3/projects/5c6e75141d254fbd82d4f1445428b223/users/afb0e1ce96d14b0492a45373bd804901/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:53.420295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6e75141d254fbd82d4f1445428b223, user_id=afb0e1ce96d14b0492a45373bd804901, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:53.423036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:53.656395 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeca440e9b524fb8b6226f6142aefd93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.657023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeca440e9b524fb8b6226f6142aefd93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.789374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeca440e9b524fb8b6226f6142aefd93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.795910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1277/2562] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:53 2022] PUT /identity/v3/projects/5c6e75141d254fbd82d4f1445428b223/users/afb0e1ce96d14b0492a45373bd804901/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:53.797526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeca440e9b524fb8b6226f6142aefd93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.799645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:53.810630 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeca440e9b524fb8b6226f6142aefd93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:53.817052 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1286/2563] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:53 2022] DELETE /identity/v3/users/aeca440e9b524fb8b6226f6142aefd93 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:53.867801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ad0be-0766-4c6e-a0fd-1a776bc47c87 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:53.929715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebcc0b-4f85-480c-acb4-e2f00dc46ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:53.930422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebcc0b-4f85-480c-acb4-e2f00dc46ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:53.930707 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebcc0b-4f85-480c-acb4-e2f00dc46ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:53.931019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebcc0b-4f85-480c-acb4-e2f00dc46ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:53.931867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebcc0b-4f85-480c-acb4-e2f00dc46ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:53.933750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebcc0b-4f85-480c-acb4-e2f00dc46ee2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:53.946306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1278/2564] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:53.950634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:53.952430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:53.967059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a21fb-7180-4085-967d-703875fea238 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:53.971723 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:53.972628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:53.972914 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:53.973192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] PATH_INFO: `/v3/projects/5c6e75141d254fbd82d4f1445428b223/users/afb0e1ce96d14b0492a45373bd804901/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:53.975379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6e75141d254fbd82d4f1445428b223, user_id=afb0e1ce96d14b0492a45373bd804901, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:53.978042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:53.988661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1287/2565] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:53 2022] PUT /identity/v3/projects/5c6e75141d254fbd82d4f1445428b223/users/afb0e1ce96d14b0492a45373bd804901/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:54.001181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:54.001501 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:54.001784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:54.010629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:54.033653 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] MFA Rules not processed for user `afb0e1ce96d14b0492a45373bd804901`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:54.034908 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:54.116607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1279/2566] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:53 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:54.128269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:54.130235 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:54.193801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce33c504-adb5-4a8c-8259-cb5cd9315cd3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:54.197748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:54.198386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:54.198672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:54.198951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:54.209939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:54.211641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:54.221611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1288/2567] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:54.348749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:54.352354 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:54.378631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:54.380486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:54.426524 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ea1e2-1362-4849-a94e-846b40c1d939 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:54.431817 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:54.432644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:54.432931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:54.433205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] PATH_INFO: `/v3/projects/fc5c41d98c6146dbb44ec38cfe82ea21` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:54.434573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5c41d98c6146dbb44ec38cfe82ea21)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:54.436623 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:54.458133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71ce2-3a0f-41bb-9441-87369b0c1bd4 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:54.464341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:54.465247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:54.465526 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:54.465804 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] PATH_INFO: `/v3/users/8f7beadcc9954c8287bbc33928a589a6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:54.467123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7beadcc9954c8287bbc33928a589a6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:54.469083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:54.625532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5c41d98c6146dbb44ec38cfe82ea21', 'request_id': 'req-24229172-eacc-4bbc-91b6-55e6b845ce79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.722216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5c41d98c6146dbb44ec38cfe82ea21', 'request_id': 'req-24229172-eacc-4bbc-91b6-55e6b845ce79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.736471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5c41d98c6146dbb44ec38cfe82ea21', 'request_id': 'req-24229172-eacc-4bbc-91b6-55e6b845ce79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.768875 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] The token cache is being invalidate because project fc5c41d98c6146dbb44ec38cfe82ea21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:54.769316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.770310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-24229172-eacc-4bbc-91b6-55e6b845ce79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.773621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1280/2568] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:54 2022] DELETE /identity/v3/projects/fc5c41d98c6146dbb44ec38cfe82ea21 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:54.782347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f561-731c-4a2f-854d-d5698ae4ae11 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:54.782690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f561-731c-4a2f-854d-d5698ae4ae11 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:54.782976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f561-731c-4a2f-854d-d5698ae4ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:54.819638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0123f561-731c-4a2f-854d-d5698ae4ae11 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:54.821713 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0123f561-731c-4a2f-854d-d5698ae4ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:54.866639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:54.887517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7beadcc9954c8287bbc33928a589a6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.892494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7beadcc9954c8287bbc33928a589a6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:54.956745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1281/2569] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:54.973308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:54.973765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:54.974149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:54.984242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7beadcc9954c8287bbc33928a589a6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:55.002661 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7beadcc9954c8287bbc33928a589a6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:55.004208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:55.005835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:55.019416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1282/2570] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:55.025399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-14b6a51d-8dd8-4296-ab90-e8967731f377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7beadcc9954c8287bbc33928a589a6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:55.028200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.034501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1289/2571] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:54 2022] DELETE /identity/v3/users/8f7beadcc9954c8287bbc33928a589a6 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:55.039992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367acdf-1e92-4c4b-b8f4-02a5cad60876 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.040636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.041140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.041493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.066002 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:55.067653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:55.094607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:55.147667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.148737 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.149161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.149570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.150740 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:55.151230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.153980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:55.169090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1290/2572] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:55.174070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1283/2573] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:55.181349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.182261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.183321 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:55.187417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:55.189032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:55.204184 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ce1e5-8bdf-4215-a238-79312cabafbb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.215075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.215932 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.216319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.216805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.218055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.221703 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:55.299919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b4958-d47d-4a27-86a5-2af68983015f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.304828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.305574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.305858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.306163 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.317751 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.325030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:55.338127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1284/2574] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:17:55.369575 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1291/2575] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:55 2022] POST /identity/v3/projects => generated 359 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:55.374581 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.392230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8373bfb-e5df-40ab-bbe8-63bb4928649d service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.484272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:55.512076 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.512963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.513255 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.513534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.514367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.517845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:55.740565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1285/2576] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:55 2022] POST /identity/v3/users => generated 280 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:17:55.745164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.760128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c1ecc-c484-426d-a9f1-bf3ec583a84c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.767844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.768608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.769122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.769591 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.770555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.772356 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:55.786495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1292/2577] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:55.791801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.805216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a41e70-45ab-4414-b29b-df16182e3052 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.811089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.811792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.812075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.812352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.813091 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.814704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:55.826959 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1286/2578] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:55.831110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:55.841949 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395fdc9-494c-4ed9-9f52-92e0a2508767 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:55.846629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8b612-30d3-4303-92fb-92f64a38de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:55.847553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8b612-30d3-4303-92fb-92f64a38de4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:55.847839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8b612-30d3-4303-92fb-92f64a38de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:55.848118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8b612-30d3-4303-92fb-92f64a38de4d admin admin] PATH_INFO: `/v3/projects/33d2ae9b01d54759a26811d353dbde97/users/49001c5fb4214699b41587d3e6f897d4/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:55.864854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8b612-30d3-4303-92fb-92f64a38de4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d2ae9b01d54759a26811d353dbde97, user_id=49001c5fb4214699b41587d3e6f897d4, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:55.867677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8b612-30d3-4303-92fb-92f64a38de4d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:56.186290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:56.188341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:56.205619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81718a5-2f7c-447f-880d-59e53af85074 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:56.212940 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:56.213850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:56.214112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:56.214417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] PATH_INFO: `/v3/projects/7adc9fbf58d74cf3a623dd8dacf1a6b8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:56.215733 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7adc9fbf58d74cf3a623dd8dacf1a6b8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:56.217595 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:56.250045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1293/2579] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:55 2022] PUT /identity/v3/projects/33d2ae9b01d54759a26811d353dbde97/users/49001c5fb4214699b41587d3e6f897d4/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:56.282361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:56.282697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:56.282913 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:56.319581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] MFA Rules not processed for user `49001c5fb4214699b41587d3e6f897d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:56.320869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:56.387233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:56.399211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1294/2580] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:56 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:56.446917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:56.446917 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:56.496901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb7e0-7f39-4a30-a8e1-77a48f61ab41 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:56.514195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7adc9fbf58d74cf3a623dd8dacf1a6b8', 'request_id': 'req-889ab88b-bf69-4a55-9382-ab12858ca011'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.556573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:56.557922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:56.558438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:56.558869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:56.625748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:56.627206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:56.637277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1295/2581] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:56 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:56.652422 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:56.692257 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7adc9fbf58d74cf3a623dd8dacf1a6b8', 'request_id': 'req-889ab88b-bf69-4a55-9382-ab12858ca011'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.701153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7adc9fbf58d74cf3a623dd8dacf1a6b8', 'request_id': 'req-889ab88b-bf69-4a55-9382-ab12858ca011'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.713598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] The token cache is being invalidate because project 7adc9fbf58d74cf3a623dd8dacf1a6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:56.713891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.714526 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.716220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1287/2582] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:56 2022] DELETE /identity/v3/projects/7adc9fbf58d74cf3a623dd8dacf1a6b8 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:56.720127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:56.721421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:56.722709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df125d-1c07-4308-98c6-cf2c03155458 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:56.728976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:56.729876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:56.730164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:56.733448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] PATH_INFO: `/v3/users/49001c5fb4214699b41587d3e6f897d4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:56.733448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] RBAC: Authorizing `identity:delete_user(user_id=49001c5fb4214699b41587d3e6f897d4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:56.733820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:56.735309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ab88b-bf69-4a55-9382-ab12858ca011 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:56.741194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:56.742082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:56.742365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:56.742643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] PATH_INFO: `/v3/users/afb0e1ce96d14b0492a45373bd804901` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:56.743896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb0e1ce96d14b0492a45373bd804901)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:56.745818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:56.970183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49001c5fb4214699b41587d3e6f897d4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.970762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49001c5fb4214699b41587d3e6f897d4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:56.998273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb0e1ce96d14b0492a45373bd804901'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.010367 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb0e1ce96d14b0492a45373bd804901'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.017291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb0e1ce96d14b0492a45373bd804901'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.098685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49001c5fb4214699b41587d3e6f897d4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.107414 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49001c5fb4214699b41587d3e6f897d4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.119814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b4aeccaf-2de0-4f42-94ce-81da6dc8833a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49001c5fb4214699b41587d3e6f897d4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.125934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1296/2583] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:56 2022] DELETE /identity/v3/users/49001c5fb4214699b41587d3e6f897d4 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:57.130407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:57.130726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:57.131121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:57.138492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:57.149263 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:57.150513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:57.189873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb0e1ce96d14b0492a45373bd804901'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.190514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb0e1ce96d14b0492a45373bd804901'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:57.198120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1288/2584] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:56 2022] DELETE /identity/v3/users/afb0e1ce96d14b0492a45373bd804901 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:57.213271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1297/2585] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:57.222770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:57.224242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:57.291128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22ebe6-0cd6-49b3-a50b-61c4a1988866 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:57.295089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:57.295855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:57.296138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:57.296414 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:57.360749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:57.362169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:57.372304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1289/2586] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:57.487046 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:57.635656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:57.809742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:57.811621 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:57.827673 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52ae7e-02f7-45b1-8d8b-92b290f6db7b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:57.832903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:57.836493 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:57.837777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:57.837970 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] PATH_INFO: `/v3/projects/5c6e75141d254fbd82d4f1445428b223` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:57.842577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6e75141d254fbd82d4f1445428b223)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:57.844612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.026125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.026125 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:58.036527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6e75141d254fbd82d4f1445428b223', 'request_id': 'req-48016500-7a53-476a-ac8c-c5f7b3a02125'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.040212 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060c5cf-e11a-45bf-a2f6-562262f36c89 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.044657 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6e75141d254fbd82d4f1445428b223', 'request_id': 'req-48016500-7a53-476a-ac8c-c5f7b3a02125'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.054954 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6e75141d254fbd82d4f1445428b223', 'request_id': 'req-48016500-7a53-476a-ac8c-c5f7b3a02125'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.111419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.112322 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.112610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.112889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] PATH_INFO: `/v3/projects/33d2ae9b01d54759a26811d353dbde97` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.114119 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d2ae9b01d54759a26811d353dbde97)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.116285 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.217933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] The token cache is being invalidate because project 5c6e75141d254fbd82d4f1445428b223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:58.218221 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.218831 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48016500-7a53-476a-ac8c-c5f7b3a02125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.220450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1298/2587] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:57 2022] DELETE /identity/v3/projects/5c6e75141d254fbd82d4f1445428b223 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:58.225384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374b6eb-bdac-42e0-86f6-74e553379a6b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.225672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374b6eb-bdac-42e0-86f6-74e553379a6b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.225953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374b6eb-bdac-42e0-86f6-74e553379a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.244852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1374b6eb-bdac-42e0-86f6-74e553379a6b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:58.246197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1374b6eb-bdac-42e0-86f6-74e553379a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:58.256648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1299/2588] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:58.266874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.266874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.266874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.279024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:58.287324 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d2ae9b01d54759a26811d353dbde97', 'request_id': 'req-8f3868fa-fc1a-4c70-8541-09462a102a2b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.290197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:58.291473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:58.356513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1300/2589] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:58.361165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.371998 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257adc4-ac2c-4fe8-90d5-18f12667b106 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.376930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.377611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.377910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.378191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.378933 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:58.379245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.381446 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.393733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1301/2590] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:17:58.398773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.409069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168db36-b0cf-4b95-bad8-6e86adf116a8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.414180 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.414854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.415145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.415454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.416162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.418104 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.452900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d2ae9b01d54759a26811d353dbde97', 'request_id': 'req-8f3868fa-fc1a-4c70-8541-09462a102a2b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.460836 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d2ae9b01d54759a26811d353dbde97', 'request_id': 'req-8f3868fa-fc1a-4c70-8541-09462a102a2b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.472170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] The token cache is being invalidate because project 33d2ae9b01d54759a26811d353dbde97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:17:58.472552 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.473248 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f3868fa-fc1a-4c70-8541-09462a102a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:58.475349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1290/2591] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:17:58 2022] DELETE /identity/v3/projects/33d2ae9b01d54759a26811d353dbde97 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:58.480626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652e4d-8412-4a87-bec2-3b6c74cc5d0b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.480626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652e4d-8412-4a87-bec2-3b6c74cc5d0b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.481048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652e4d-8412-4a87-bec2-3b6c74cc5d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.501957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-35652e4d-8412-4a87-bec2-3b6c74cc5d0b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:58.503189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-35652e4d-8412-4a87-bec2-3b6c74cc5d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:58.512452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1291/2592] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:58.520400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.520700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.520987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.543627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:58.543858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:58.552218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1292/2593] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:17:58.556981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.567656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9795ba-39de-4266-b037-aa315e9bd9f1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.573387 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.574175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.575106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.575106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.575592 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:17:58.575905 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.577974 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.598633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1293/2594] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:17:58.600432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.610644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d8744-504a-410c-a0f2-f49c32c26ba1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.616021 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.616804 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.617136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.617523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.618316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.620460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.652358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1302/2595] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/projects => generated 403 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:58.655280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.655280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:58.722926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055632a-427f-4821-ab9c-b6bde0870e03 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.729061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.729762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.730047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.730384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.731610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.733420 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.741589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1294/2596] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:58.745401 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.815469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323b67a-fe04-4760-ad4c-af7ae8cf4dee admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.821884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.827479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.827479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.827479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.827479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.827479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.860390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1303/2597] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/users => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:58.865381 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.876495 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365afab-be89-4705-8bb4-fb64d454e7eb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.881996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.882810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.883028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.883305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.884528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.885757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.899260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1304/2598] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:58.903856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.915852 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dd707-e9e4-418c-a6c7-088e1f173460 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.921823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.922551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.922840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.923123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.923926 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.925656 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:58.940162 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1305/2599] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:17:58.944785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:58.958204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40125d0-9629-428a-b5c8-72a4b4fb81d4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:58.964688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ee725-aca5-42cb-aa52-83e8b97a94ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:58.965641 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ee725-aca5-42cb-aa52-83e8b97a94ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:58.965989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ee725-aca5-42cb-aa52-83e8b97a94ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:58.966270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ee725-aca5-42cb-aa52-83e8b97a94ea admin admin] PATH_INFO: `/v3/projects/0072eac27d6e471d99730302fa5d86fa/users/7b5b33f989dc43fbbef1349cd7464d64/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:58.985201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ee725-aca5-42cb-aa52-83e8b97a94ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0072eac27d6e471d99730302fa5d86fa, user_id=7b5b33f989dc43fbbef1349cd7464d64, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:58.987938 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ee725-aca5-42cb-aa52-83e8b97a94ea admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.026697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1295/2600] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:17:58 2022] POST /identity/v3/users => generated 287 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:17:59.030925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.042235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8f4f6-cfed-4f08-81c9-915f5b0c8d32 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:59.052227 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:59.053499 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.053919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.054396 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.055105 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:59.056907 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.070162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1296/2601] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:59.074448 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.086077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da527f72-8a47-4b8c-88ab-068cc7ebd348 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:59.097393 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:59.099138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.099138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.099699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.100100 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:59.104973 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.114578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1297/2602] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:17:59.119281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.134576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22111-5e8d-4114-844e-a75d2bfd6226 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:59.140026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bd8e6-6247-490b-99da-7be3564140fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:59.141147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bd8e6-6247-490b-99da-7be3564140fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.141546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bd8e6-6247-490b-99da-7be3564140fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.149731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bd8e6-6247-490b-99da-7be3564140fd admin admin] PATH_INFO: `/v3/projects/c5f1f9df287e436998deb85ef6ea7386/users/824d6fbe6b9a4f0fa4cce9e14d5ffad7/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.161015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bd8e6-6247-490b-99da-7be3564140fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f1f9df287e436998deb85ef6ea7386, user_id=824d6fbe6b9a4f0fa4cce9e14d5ffad7, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:59.163746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bd8e6-6247-490b-99da-7be3564140fd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.213551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1306/2603] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:58 2022] PUT /identity/v3/projects/0072eac27d6e471d99730302fa5d86fa/users/7b5b33f989dc43fbbef1349cd7464d64/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:17:59.218876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.219182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.219503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.237788 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:59.239051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:59.255226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:17:59.312509 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1307/2604] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:59.327560 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.328974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:59.373687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1298/2605] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:17:59 2022] PUT /identity/v3/projects/c5f1f9df287e436998deb85ef6ea7386/users/824d6fbe6b9a4f0fa4cce9e14d5ffad7/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:17:59.384607 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.384898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.385176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.393572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5b9f4-4c4d-4942-b691-fdbb4d0b74ec None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:59.414217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] MFA Rules not processed for user `824d6fbe6b9a4f0fa4cce9e14d5ffad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:17:59.415479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:59.446889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:59.447579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.455733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.455733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.487238 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1299/2606] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:17:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:17:59.500669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.513290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:59.519921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:59.522573 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.535524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1308/2607] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:59.573546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a371d8-e330-4cf2-a77e-c08a9f76551d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:59.580192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:59.580816 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.581180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.581384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.597696 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:59.600493 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.613409 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1300/2608] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:17:59.640501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.651905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00a173-7edc-4d6b-8229-9fe9576b2110 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:17:59.657296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:17:59.658282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:17:59.658566 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:17:59.658837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] PATH_INFO: `/v3/users/824d6fbe6b9a4f0fa4cce9e14d5ffad7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:17:59.660114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=824d6fbe6b9a4f0fa4cce9e14d5ffad7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:17:59.662178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:17:59.878553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:17:59.880361 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:17:59.995239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824d6fbe6b9a4f0fa4cce9e14d5ffad7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:17:59.997023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824d6fbe6b9a4f0fa4cce9e14d5ffad7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:00.015579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d91a04-5be3-40a3-afe6-300dbd00a4dc service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:00.023938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:00.024688 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:00.024941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:00.025221 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:00.028852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:00.048571 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:00.050193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de77f1-01ce-43e8-b7a0-948e5e11881b service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:00.064828 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1301/2609] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:17:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:18:00.365601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824d6fbe6b9a4f0fa4cce9e14d5ffad7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:00.372923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824d6fbe6b9a4f0fa4cce9e14d5ffad7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:00.418111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824d6fbe6b9a4f0fa4cce9e14d5ffad7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:00.432976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1309/2610] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:17:59 2022] DELETE /identity/v3/users/824d6fbe6b9a4f0fa4cce9e14d5ffad7 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:00.437934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:00.438137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:00.438416 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:00.458090 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:00.460419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:00.523080 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:00.544058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1302/2611] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:00.556632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:00.558076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:00.627797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c99fe-09ac-4d97-97c2-fcdc6d8be0aa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:00.632730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f857d8-0848-46be-b56e-96df5d8b3a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:00.633423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f857d8-0848-46be-b56e-96df5d8b3a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:00.633712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f857d8-0848-46be-b56e-96df5d8b3a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:00.634055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f857d8-0848-46be-b56e-96df5d8b3a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:00.649517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f857d8-0848-46be-b56e-96df5d8b3a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:00.651515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f857d8-0848-46be-b56e-96df5d8b3a79 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:00.666901 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1310/2612] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:01.754231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:01.756456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:01.769418 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24867a01-2343-4dc9-98bd-18a24a0e3cf4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:01.775980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:01.777329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:01.777853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:01.778351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] PATH_INFO: `/v3/projects/c5f1f9df287e436998deb85ef6ea7386` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:01.780576 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f1f9df287e436998deb85ef6ea7386)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:01.782718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:02.055645 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f1f9df287e436998deb85ef6ea7386', 'request_id': 'req-3affbb82-f3a9-4c9e-803d-572082f07467'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:02.344457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f1f9df287e436998deb85ef6ea7386', 'request_id': 'req-3affbb82-f3a9-4c9e-803d-572082f07467'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:02.352830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f1f9df287e436998deb85ef6ea7386', 'request_id': 'req-3affbb82-f3a9-4c9e-803d-572082f07467'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:02.363808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] The token cache is being invalidate because project c5f1f9df287e436998deb85ef6ea7386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:02.364098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:02.364753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3affbb82-f3a9-4c9e-803d-572082f07467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:02.367300 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1303/2613] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:01 2022] DELETE /identity/v3/projects/c5f1f9df287e436998deb85ef6ea7386 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:02.372537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.373121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.373379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.394270 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:02.395604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:02.471826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1311/2614] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:02.481923 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164008c-ec32-4b9d-ac81-d78bc9e3b525 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.482220 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164008c-ec32-4b9d-ac81-d78bc9e3b525 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.482508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164008c-ec32-4b9d-ac81-d78bc9e3b525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.503664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b164008c-ec32-4b9d-ac81-d78bc9e3b525 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:02.504917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b164008c-ec32-4b9d-ac81-d78bc9e3b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:02.514686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1304/2615] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:02.519399 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:02.531639 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a9f14-2e86-4932-94e0-a02977747287 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:02.540517 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:02.541714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.542109 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.542270 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.542542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.542726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.543616 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.545056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:02.545475 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:02.547961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:02.560877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1312/2616] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:02.564870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:02.572099 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] MFA Rules not processed for user `7b5b33f989dc43fbbef1349cd7464d64`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:02.573351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:02.574820 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f230e2-4ebf-4e6b-9caa-2507509b4585 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:02.580015 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:02.580685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.580974 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.581258 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.582248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:02.584745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:02.649178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1305/2617] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:02.653668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:02.654990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:02.669081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1cc17-3d87-43fd-af3c-8aabed2bd48c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:02.674810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:02.675602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.678470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.678470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.678470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:02.679760 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:02.723456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1313/2618] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:02.728636 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:02.796822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b47a2a-c278-406d-b399-c68c5cc81e8b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:02.803264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:02.804369 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.804777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.805131 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.805964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:02.808036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:02.860381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1306/2619] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/projects => generated 405 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:02.864266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:02.940319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6881133f-6b3d-438c-9403-f1441c272401 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:02.945796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:02.946535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:02.946870 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:02.947150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:02.948062 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:02.950000 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.053725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1314/2620] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/users => generated 296 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:03.058727 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.073922 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0536f53-fab0-4827-8a10-c046e84c19e8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.076062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.077122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.077412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.077694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.078445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.080218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.092241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1315/2621] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:03.096355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.104584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:03.109274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d9b1d-22de-4544-af72-55c2592215ca admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.123327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.123327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.123327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.123327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.124093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.124093 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.133707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1316/2622] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:03.137592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.148430 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfb813-5dd5-497b-8e49-34071df981cb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.153146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a04cc1-78af-4e85-8f2f-b20a9837f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.154034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a04cc1-78af-4e85-8f2f-b20a9837f2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.154317 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a04cc1-78af-4e85-8f2f-b20a9837f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.154593 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a04cc1-78af-4e85-8f2f-b20a9837f2f2 admin admin] PATH_INFO: `/v3/projects/f0110f17b4ca48c88ff42f73c0f7e880/users/0ed3b91fed9f4338b4d428bd65a518cb/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.170207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a04cc1-78af-4e85-8f2f-b20a9837f2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0110f17b4ca48c88ff42f73c0f7e880, user_id=0ed3b91fed9f4338b4d428bd65a518cb, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.172985 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a04cc1-78af-4e85-8f2f-b20a9837f2f2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.190191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1307/2623] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:02 2022] POST /identity/v3/users => generated 309 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:03.194081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.204072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4214756-b9d9-4220-a9d9-61a6867ae29d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.209624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.210295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.210618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.211026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.211756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.213372 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.225007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1308/2624] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:03.229507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.240521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816548cd-bdfa-4d13-b0f5-1800c7aad03a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.245464 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.246379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.246910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.247161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] PATH_INFO: `/v3/projects/3b1d0a3464514f33b919afe6b0a99b7f/users/cf7caae683b243a19f69c88a26773775/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.265244 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1d0a3464514f33b919afe6b0a99b7f, user_id=cf7caae683b243a19f69c88a26773775, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.268015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.362178 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:03.370577 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1317/2625] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:03 2022] PUT /identity/v3/projects/f0110f17b4ca48c88ff42f73c0f7e880/users/0ed3b91fed9f4338b4d428bd65a518cb/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:03.382147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.382792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.382792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.412054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] MFA Rules not processed for user `0ed3b91fed9f4338b4d428bd65a518cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:03.413535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:03.453403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1309/2626] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:03 2022] PUT /identity/v3/projects/3b1d0a3464514f33b919afe6b0a99b7f/users/cf7caae683b243a19f69c88a26773775/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:03.461230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.485511 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1318/2627] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:03.497332 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.498992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:03.526415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9363-0f4d-456f-9976-861da5015c1e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.533999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.535182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.535831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.536182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.537554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.540370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.563577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1310/2628] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:03.565450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:03.584458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf6de0-e3ab-4e9b-87fb-f0ea9e246d40 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.590294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a8313-e602-499f-a371-3ae7d2f5b28c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:03.592795 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbb2d3-a208-4b27-a0e7-1f9f36b5c9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.594261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbb2d3-a208-4b27-a0e7-1f9f36b5c9a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.594853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbb2d3-a208-4b27-a0e7-1f9f36b5c9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.595302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbb2d3-a208-4b27-a0e7-1f9f36b5c9a8 admin admin] PATH_INFO: `/v3/projects/3b1d0a3464514f33b919afe6b0a99b7f/users/cf7caae683b243a19f69c88a26773775/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.602933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbb2d3-a208-4b27-a0e7-1f9f36b5c9a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1d0a3464514f33b919afe6b0a99b7f, user_id=cf7caae683b243a19f69c88a26773775, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.603288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbb2d3-a208-4b27-a0e7-1f9f36b5c9a8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.619684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1311/2629] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:03 2022] PUT /identity/v3/projects/3b1d0a3464514f33b919afe6b0a99b7f/users/cf7caae683b243a19f69c88a26773775/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:03.654549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:03.654549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:03.654549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:03.654549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:03.726140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:03.727757 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:03.737561 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1319/2630] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:18:04.842730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:04.990601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:05.314941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:05.314941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:05.314941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:05.345266 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] MFA Rules not processed for user `cf7caae683b243a19f69c88a26773775`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:05.346750 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:05.430353 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1312/2631] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:05 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:06.852529 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:06.853899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:06.854508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:06.905148 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:06.921448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816c0d5-84a5-483f-b3c4-7e29d313d328 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:06.987327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:06.988169 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:06.988524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:06.988870 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:06.992195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:07.064158 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:07.065609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:07.080079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1320/2632] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:07.098579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:07.111141 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80cfef-9f76-48c1-bd8d-32b2551c9128 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:07.117718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:07.118608 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:07.119124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:07.119560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] PATH_INFO: `/v3/users/7b5b33f989dc43fbbef1349cd7464d64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:07.121953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5b33f989dc43fbbef1349cd7464d64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:07.124150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:07.418032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5b33f989dc43fbbef1349cd7464d64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:07.433567 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5b33f989dc43fbbef1349cd7464d64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:07.447560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5b33f989dc43fbbef1349cd7464d64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:07.745077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5b33f989dc43fbbef1349cd7464d64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:07.746172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5b33f989dc43fbbef1349cd7464d64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:07.754403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1313/2633] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:07 2022] DELETE /identity/v3/users/7b5b33f989dc43fbbef1349cd7464d64 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:08.100666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:08.113614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73367a-21e7-4633-a0e3-37d53338e648 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:08.193243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:08.193243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:08.193243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:08.193243 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] PATH_INFO: `/v3/projects/0072eac27d6e471d99730302fa5d86fa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:08.193941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0072eac27d6e471d99730302fa5d86fa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:08.194670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:08.259056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:08.330911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0072eac27d6e471d99730302fa5d86fa', 'request_id': 'req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.337376 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0072eac27d6e471d99730302fa5d86fa', 'request_id': 'req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.343891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0072eac27d6e471d99730302fa5d86fa', 'request_id': 'req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.475966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] The token cache is being invalidate because project 0072eac27d6e471d99730302fa5d86fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:08.482097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.482097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.482097 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1321/2634] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:08 2022] DELETE /identity/v3/projects/0072eac27d6e471d99730302fa5d86fa => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:08.482916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:08.484081 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:08.502228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de045674-a880-4109-85f9-422912a9b077 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:08.572362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:08.573265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:08.573553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:08.573833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] PATH_INFO: `/v3/users/cf7caae683b243a19f69c88a26773775` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:08.575133 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf7caae683b243a19f69c88a26773775)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:08.577448 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:08.744968 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:08.829269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf7caae683b243a19f69c88a26773775'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.844450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf7caae683b243a19f69c88a26773775'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.852577 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf7caae683b243a19f69c88a26773775'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.932742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf7caae683b243a19f69c88a26773775'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.935035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-08972667-555b-4475-a5be-730e279b3afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf7caae683b243a19f69c88a26773775'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:08.946175 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1314/2635] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:08 2022] DELETE /identity/v3/users/cf7caae683b243a19f69c88a26773775 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:09.360431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:09.362168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:09.377784 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4986b0-4c9c-47a1-b838-d87e14b15a2c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:09.432364 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:09.467745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:09.468647 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:09.468944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:09.469220 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] PATH_INFO: `/v3/projects/3b1d0a3464514f33b919afe6b0a99b7f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:09.471693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1d0a3464514f33b919afe6b0a99b7f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:09.475108 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:09.641714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1d0a3464514f33b919afe6b0a99b7f', 'request_id': 'req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:09.649039 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1d0a3464514f33b919afe6b0a99b7f', 'request_id': 'req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:09.656441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1d0a3464514f33b919afe6b0a99b7f', 'request_id': 'req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:09.749768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] The token cache is being invalidate because project 3b1d0a3464514f33b919afe6b0a99b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:09.750059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:09.751464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e4ed0946-7ff4-4ec2-aacd-b1a5fa221728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:09.753456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1322/2636] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:09 2022] DELETE /identity/v3/projects/3b1d0a3464514f33b919afe6b0a99b7f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:09.760312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:09.760871 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:09.760995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:09.780587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:09.781821 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:09.853223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1315/2637] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:09.862667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:09.863044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:09.866622 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:09.885799 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:09.887266 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:09.901677 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1323/2638] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:09.907612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:09.924793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e38f-796f-4d7b-8565-6bf4637297f0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:09.935629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:09.936699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:09.937208 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:09.937544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:09.938403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:09.938956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:09.941654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:09.946203 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:09.960454 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1316/2639] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:09.965872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:09.978319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b0324-87de-472a-8f44-5b71b1e45185 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:09.985989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:09.986809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:09.987194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:09.987571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:09.988355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:09.990655 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.093741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1324/2640] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:09 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:10.094914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:10.108882 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88209d2-a304-46b6-b6c1-eb6b2f6aab7b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:10.182532 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:10.183301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.188500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.188500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.188500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:10.188500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.355785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1317/2641] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:10 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:10.357034 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:10.369476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba1f1e-bcbe-4604-9265-c14e3810f9c4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:10.376521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:10.377523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.378045 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.378556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.379485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:10.381953 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.399800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1325/2642] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:10.403630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:10.414667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a43ff-0ac0-41fb-9fe9-4456bab3a967 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:10.419986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577825b-affd-44b2-86a4-58f4952173dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:10.422857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577825b-affd-44b2-86a4-58f4952173dd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.422857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577825b-affd-44b2-86a4-58f4952173dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.422857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577825b-affd-44b2-86a4-58f4952173dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.422857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577825b-affd-44b2-86a4-58f4952173dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:10.424025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577825b-affd-44b2-86a4-58f4952173dd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.435970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1318/2643] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:10.439918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:10.453535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c8c41-1640-403c-a468-feb2c4c5776b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:10.460536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:10.463190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.463521 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.464911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] PATH_INFO: `/v3/projects/ed77c486cfa647e3b2c7c845d9bc9d77/users/a22e922b3ba64d3a983e7d18c6f66b1f/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.486579 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed77c486cfa647e3b2c7c845d9bc9d77, user_id=a22e922b3ba64d3a983e7d18c6f66b1f, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:10.501141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.598361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1326/2644] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:10 2022] PUT /identity/v3/projects/ed77c486cfa647e3b2c7c845d9bc9d77/users/a22e922b3ba64d3a983e7d18c6f66b1f/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:10.610112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.610309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.610589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.643625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] MFA Rules not processed for user `a22e922b3ba64d3a983e7d18c6f66b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:10.644886 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:10.747975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1319/2645] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:10.748552 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:10.897756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837777a0-d33f-431e-93de-6e6eef175b4b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:10.903676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:10.906372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:10.906372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:10.906372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:10.906372 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:10.908208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:10.917811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:11.053303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1327/2646] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:10 2022] POST /identity/v3/projects => generated 367 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:11.055069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.073237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e398234-4cb5-4cd2-8c60-68a63abd553a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.147933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.148458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.148735 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.149003 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.149717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.151703 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.161008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:11.317002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1320/2647] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:11 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:11.321207 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.334916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288f8a4-ef1b-4987-8c15-b6bc15a35d41 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.340196 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.341157 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.341547 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.342146 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.343252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.345123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.368486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1328/2648] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:11.372793 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.384800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efb07a-10ae-48eb-9c00-0ed4bba3ce43 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.390379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.391359 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.391683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.392128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] PATH_INFO: `/v3/projects/813bbf8b517e43b9b52d32f4eaff2d11/users/612971f8d9fe4d1d8b0484994bc3a6cd/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.419058 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813bbf8b517e43b9b52d32f4eaff2d11, user_id=612971f8d9fe4d1d8b0484994bc3a6cd, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.421785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.550787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1321/2649] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:11 2022] PUT /identity/v3/projects/813bbf8b517e43b9b52d32f4eaff2d11/users/612971f8d9fe4d1d8b0484994bc3a6cd/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:11.556309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.567548 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a593-8846-42c3-8aa1-92d5d455593e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.644909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.645781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.646147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.646730 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.647694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.649350 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.670586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.672398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:11.679013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1329/2650] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:11.680500 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.689824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e04152-8586-4e9e-b60d-3ff8d2908df2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.692694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75986eb-8bdd-4f1e-adbd-d3c84d92e0e9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.695722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f99131-1819-488e-925e-5082214f222d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.696612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f99131-1819-488e-925e-5082214f222d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.696902 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f99131-1819-488e-925e-5082214f222d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.697182 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f99131-1819-488e-925e-5082214f222d admin admin] PATH_INFO: `/v3/users/30ac12728e8f4ec9af534612030d75eb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.698584 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f99131-1819-488e-925e-5082214f222d admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ac12728e8f4ec9af534612030d75eb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.700689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda41eb-e04d-44c3-9593-f63958f28a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.700834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f99131-1819-488e-925e-5082214f222d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.702968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda41eb-e04d-44c3-9593-f63958f28a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.703453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda41eb-e04d-44c3-9593-f63958f28a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.703940 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda41eb-e04d-44c3-9593-f63958f28a73 admin admin] PATH_INFO: `/v3/projects/813bbf8b517e43b9b52d32f4eaff2d11/users/612971f8d9fe4d1d8b0484994bc3a6cd/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.707250 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda41eb-e04d-44c3-9593-f63958f28a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813bbf8b517e43b9b52d32f4eaff2d11, user_id=612971f8d9fe4d1d8b0484994bc3a6cd, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:11.712687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda41eb-e04d-44c3-9593-f63958f28a73 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:11.736429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1330/2651] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:11 2022] PUT /identity/v3/projects/813bbf8b517e43b9b52d32f4eaff2d11/users/612971f8d9fe4d1d8b0484994bc3a6cd/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:11.748997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.758273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.758273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:11.780555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] MFA Rules not processed for user `612971f8d9fe4d1d8b0484994bc3a6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:11.781851 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:11.873602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1331/2652] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:11 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:11.890073 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:11.891586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:11.920738 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ac12728e8f4ec9af534612030d75eb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:11.939242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ac12728e8f4ec9af534612030d75eb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:11.951123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ac12728e8f4ec9af534612030d75eb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:11.979474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:11.985025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2c04b-a4cf-48c5-abb5-4c7eb0324ac2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:11.989024 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:11.991942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:11.991942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:11.991942 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:12.000818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:12.002335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:12.013837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1332/2653] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:18:12.052822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ac12728e8f4ec9af534612030d75eb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.053622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ac12728e8f4ec9af534612030d75eb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.060843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1322/2654] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:11 2022] DELETE /identity/v3/users/30ac12728e8f4ec9af534612030d75eb => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:12.298646 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:12.308323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f961d-0e4b-4c1b-a617-549205337d74 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:12.361367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:12.362124 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:12.362551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:12.362879 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:12.365343 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:12.434050 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:12.435734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2939c44-746c-4744-8fc8-9137c9e5c484 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:12.446272 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1333/2655] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:12.452854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:12.455218 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:12.569086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f99131-1819-488e-925e-5082214f222d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:12.574190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:12.575079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:12.578206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:12.578206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] PATH_INFO: `/v3/projects/11089493792047939c9f45f632a0ce05` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:12.578206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11089493792047939c9f45f632a0ce05)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:12.579153 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:12.720618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=3,keystone:SELECT=84,keystone:DELETE=13 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:12.726472 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11089493792047939c9f45f632a0ce05', 'request_id': 'req-389206d3-fa0e-4f12-91cf-e605730d69b2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.852819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11089493792047939c9f45f632a0ce05', 'request_id': 'req-389206d3-fa0e-4f12-91cf-e605730d69b2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.861953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11089493792047939c9f45f632a0ce05', 'request_id': 'req-389206d3-fa0e-4f12-91cf-e605730d69b2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.875005 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] The token cache is being invalidate because project 11089493792047939c9f45f632a0ce05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:12.875307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.876179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-389206d3-fa0e-4f12-91cf-e605730d69b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:12.877753 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1323/2656] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:12 2022] DELETE /identity/v3/projects/11089493792047939c9f45f632a0ce05 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:12.885083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:12.893347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:12.893347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:12.912296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:12.913610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:12.978088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1334/2657] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:12.986859 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:12.987130 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:12.987442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.009376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:13.010624 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:13.027279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1324/2658] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:13.039208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.051471 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8565c-f7a8-4d4f-99e9-28b66f2ac471 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.056903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.057704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.057993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.058277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.059012 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:13.059325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:13.060975 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:13.074781 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1335/2659] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:13.080066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.090415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c9f47-2b5b-4f56-9cb5-b063b2ed8e73 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.095292 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.097787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.097787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.097787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.097787 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:13.099511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:13.249554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1325/2660] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:13 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:13.262611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.273722 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c759c-6efc-4843-8790-e2b1eee1d172 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.305344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:13.337346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.338174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.338607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.339644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.341667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:13.341667 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:13.507030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1336/2661] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:13 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:13.511236 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.523071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe91b54-d4fb-4f6b-9aa0-f1959325c86d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.530310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.530310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.530310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.530310 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.530697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:13.531771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:13.546986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1326/2662] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:13.549366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.560423 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabee74-8b60-45e7-88e2-7cd2d49557fa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.565340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c316f-1d16-435f-86e3-948327b63672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.566000 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c316f-1d16-435f-86e3-948327b63672 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.567713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c316f-1d16-435f-86e3-948327b63672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.567713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c316f-1d16-435f-86e3-948327b63672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.567713 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c316f-1d16-435f-86e3-948327b63672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:13.568951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c316f-1d16-435f-86e3-948327b63672 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:13.580960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1337/2663] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:13.588158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.598349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb63193-d5ee-4138-abfe-2a197e45a33b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.605199 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.606139 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.606421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.606714 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] PATH_INFO: `/v3/projects/bc410a7799824e04893aa6d547228687/users/1dca25b8a07f46bab05f8574cbeb56e7/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.627890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc410a7799824e04893aa6d547228687, user_id=1dca25b8a07f46bab05f8574cbeb56e7, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:13.629268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:13.763047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1327/2664] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:13 2022] PUT /identity/v3/projects/bc410a7799824e04893aa6d547228687/users/1dca25b8a07f46bab05f8574cbeb56e7/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:13.774071 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.774361 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.774793 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:13.806442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] MFA Rules not processed for user `1dca25b8a07f46bab05f8574cbeb56e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:13.807853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:13.878682 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1338/2665] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:13.891138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:13.893168 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:13.982235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76739d-c2f8-4ff1-85f7-4d3afacc7251 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:13.988050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98bdf5-b3a1-4a21-b4b3-b4feca8d804a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:13.988716 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bdf5-b3a1-4a21-b4b3-b4feca8d804a service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:13.989007 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bdf5-b3a1-4a21-b4b3-b4feca8d804a service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:13.989281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98bdf5-b3a1-4a21-b4b3-b4feca8d804a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:14.000004 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bdf5-b3a1-4a21-b4b3-b4feca8d804a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:14.003041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98bdf5-b3a1-4a21-b4b3-b4feca8d804a service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:14.012686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1328/2666] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:23.875701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:24.008375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=69,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:30.400699 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:30.402244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:30.477589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5722cda-c58b-445e-8764-c4eb9389ece3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:30.482469 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:30.483531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:30.483817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:30.484091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] PATH_INFO: `/v3/users/a22e922b3ba64d3a983e7d18c6f66b1f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:30.485344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22e922b3ba64d3a983e7d18c6f66b1f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:30.487288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:30.746444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22e922b3ba64d3a983e7d18c6f66b1f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:30.747358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22e922b3ba64d3a983e7d18c6f66b1f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:30.961398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22e922b3ba64d3a983e7d18c6f66b1f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:30.969785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22e922b3ba64d3a983e7d18c6f66b1f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:30.983896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22e922b3ba64d3a983e7d18c6f66b1f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:30.990635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1339/2667] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:30 2022] DELETE /identity/v3/users/a22e922b3ba64d3a983e7d18c6f66b1f => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:30.996088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:30.999625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:30.999625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:31.017600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:31.019134 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:31.104331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1329/2668] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:31.115697 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:31.117299 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:31.185139 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ecdec-6378-42f8-9a0c-e1e80add0f6e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:31.189476 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:31.190144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:31.190433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:31.190714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:31.200705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:31.202265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:31.213313 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1340/2669] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:31.741006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:31.742845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:31.758596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa9fe-a66d-4403-983a-077730582982 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:31.765788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:31.767269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:31.768054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:31.768565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] PATH_INFO: `/v3/projects/ed77c486cfa647e3b2c7c845d9bc9d77` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:31.770635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed77c486cfa647e3b2c7c845d9bc9d77)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:31.772980 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:31.941295 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed77c486cfa647e3b2c7c845d9bc9d77', 'request_id': 'req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.046537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed77c486cfa647e3b2c7c845d9bc9d77', 'request_id': 'req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.055762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed77c486cfa647e3b2c7c845d9bc9d77', 'request_id': 'req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.069201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] The token cache is being invalidate because project ed77c486cfa647e3b2c7c845d9bc9d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:32.070027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.070764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.072844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1330/2670] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:31 2022] DELETE /identity/v3/projects/ed77c486cfa647e3b2c7c845d9bc9d77 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:32.078083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:32.090204 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39f6fa-345c-4d45-9bd4-0284c0247699 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:32.112325 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:32.168065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:32.168065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.168065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.168065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] PATH_INFO: `/v3/users/612971f8d9fe4d1d8b0484994bc3a6cd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.174206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612971f8d9fe4d1d8b0484994bc3a6cd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:32.174206 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:32.276087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:32.278747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:32.301444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d8b63-4896-4dd6-8b66-7e90263c9fc3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:32.304559 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:32.306111 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.307180 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.307691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] PATH_INFO: `/v3/users/1dca25b8a07f46bab05f8574cbeb56e7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.309916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dca25b8a07f46bab05f8574cbeb56e7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:32.313488 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:32.466289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612971f8d9fe4d1d8b0484994bc3a6cd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.467353 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612971f8d9fe4d1d8b0484994bc3a6cd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.555555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612971f8d9fe4d1d8b0484994bc3a6cd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.565615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612971f8d9fe4d1d8b0484994bc3a6cd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.579887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5dc0cfd8-e69b-4dd0-aa23-50ba02b452f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612971f8d9fe4d1d8b0484994bc3a6cd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.586217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1341/2671] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:32 2022] DELETE /identity/v3/users/612971f8d9fe4d1d8b0484994bc3a6cd => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:32.612191 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dca25b8a07f46bab05f8574cbeb56e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.630591 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dca25b8a07f46bab05f8574cbeb56e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.644596 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dca25b8a07f46bab05f8574cbeb56e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.654408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:32.824651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dca25b8a07f46bab05f8574cbeb56e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.825308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dca25b8a07f46bab05f8574cbeb56e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:32.832925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1331/2672] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:32 2022] DELETE /identity/v3/users/1dca25b8a07f46bab05f8574cbeb56e7 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:32.843886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:32.844200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:32.844698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:32.864192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:32.865467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:32.954187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1342/2673] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:32.965350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:32.970770 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:33.081712 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4bd08-fb51-4904-9743-dee2c82a3a81 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.087308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53bc62-e6e9-4944-8f7c-ae464c3fdafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.090023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bc62-e6e9-4944-8f7c-ae464c3fdafd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.090023 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bc62-e6e9-4944-8f7c-ae464c3fdafd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.090475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bc62-e6e9-4944-8f7c-ae464c3fdafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.108684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bc62-e6e9-4944-8f7c-ae464c3fdafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.111235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bc62-e6e9-4944-8f7c-ae464c3fdafd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.130509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1332/2674] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:33.255059 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.262074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:33.285709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fb272-c019-490d-bfff-0fc5da8b06b3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.294777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.296341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.296827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.297589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] PATH_INFO: `/v3/projects/813bbf8b517e43b9b52d32f4eaff2d11` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.300020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=813bbf8b517e43b9b52d32f4eaff2d11)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.304155 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.547991 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:33.564022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813bbf8b517e43b9b52d32f4eaff2d11', 'request_id': 'req-18921da7-7cc6-48fe-9edb-87c2166ff07c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:33.579801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813bbf8b517e43b9b52d32f4eaff2d11', 'request_id': 'req-18921da7-7cc6-48fe-9edb-87c2166ff07c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:33.588590 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813bbf8b517e43b9b52d32f4eaff2d11', 'request_id': 'req-18921da7-7cc6-48fe-9edb-87c2166ff07c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:33.696944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] The token cache is being invalidate because project 813bbf8b517e43b9b52d32f4eaff2d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:33.697439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:33.698589 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:33.700911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1343/2675] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:33 2022] DELETE /identity/v3/projects/813bbf8b517e43b9b52d32f4eaff2d11 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:33.708389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32986d5-6d85-419c-912a-fdb941da0d0b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.708767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32986d5-6d85-419c-912a-fdb941da0d0b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.709108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32986d5-6d85-419c-912a-fdb941da0d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.729702 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d32986d5-6d85-419c-912a-fdb941da0d0b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:33.730960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d32986d5-6d85-419c-912a-fdb941da0d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:33.796455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.802136 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:33.856196 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1333/2676] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:33.868739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.876456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.876917 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.908620 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:33.912289 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:33.922380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18921da7-7cc6-48fe-9edb-87c2166ff07c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.924400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1334/2677] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:33.929413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.935708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.937282 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.937711 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.938223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] PATH_INFO: `/v3/projects/bc410a7799824e04893aa6d547228687` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.940600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc410a7799824e04893aa6d547228687)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.942469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78cf26-d32e-4bdd-96d9-7c5bf5b20811 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:33.945185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.948364 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:33.949304 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:33.949768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:33.950400 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:33.951331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:33.951855 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:33.954071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:33.970035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1335/2678] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:33.975432 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:33.991732 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03e007-d73d-48d2-bd0c-6842b5a2e668 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.008686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.009548 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.009844 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.010128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.010851 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.013041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.109966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc410a7799824e04893aa6d547228687', 'request_id': 'req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.122248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc410a7799824e04893aa6d547228687', 'request_id': 'req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.132314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc410a7799824e04893aa6d547228687', 'request_id': 'req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.140631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1336/2679] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:33 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:34.142278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.144027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.240693 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:34.253340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f311fe-c40b-4bb7-9873-5bb20474164e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.260450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.261741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.262037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.262314 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] PATH_INFO: `/v3/users/0ed3b91fed9f4338b4d428bd65a518cb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.263771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed3b91fed9f4338b4d428bd65a518cb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.265767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.317062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] The token cache is being invalidate because project bc410a7799824e04893aa6d547228687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:34.317768 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.318816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.321693 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1344/2680] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:33 2022] DELETE /identity/v3/projects/bc410a7799824e04893aa6d547228687 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:34.323289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.328770 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.444634 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:34.451154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1f3e1-d5e7-4788-be67-6e507dedb42d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.457195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e4380-2291-4adf-91ef-ab6f3da77e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.457905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4380-2291-4adf-91ef-ab6f3da77e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.458205 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4380-2291-4adf-91ef-ab6f3da77e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.458496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4380-2291-4adf-91ef-ab6f3da77e49 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.459268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e4380-2291-4adf-91ef-ab6f3da77e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.461380 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e4380-2291-4adf-91ef-ab6f3da77e49 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.568705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1345/2681] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:34 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:34.570121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.570411 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.570802 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.593582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:34.594865 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.605540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1346/2682] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:34.605540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.616960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4614c1-ec91-4a24-a620-11a3fb097b3f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.624345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcadd0c-701f-43fa-88bc-bfc135667aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.624345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadd0c-701f-43fa-88bc-bfc135667aec admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.624731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadd0c-701f-43fa-88bc-bfc135667aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.624990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadd0c-701f-43fa-88bc-bfc135667aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.625797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcadd0c-701f-43fa-88bc-bfc135667aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.627606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcadd0c-701f-43fa-88bc-bfc135667aec admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.647161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1347/2683] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:34.647161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.647161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.647161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.650876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed3b91fed9f4338b4d428bd65a518cb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.673316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed3b91fed9f4338b4d428bd65a518cb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.678241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:34.679800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.686981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed3b91fed9f4338b4d428bd65a518cb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.699830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1348/2684] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:34.701174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.718836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18cce7-e20c-4b6d-abab-d3b668a7c85b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.727265 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.729144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.729698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.730168 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.731592 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.733883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.748732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1349/2685] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:34.750729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.762359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9858f50-2375-4fc9-ae62-58959476e405 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.768473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.769461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.769849 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.770191 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.771140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:34.773122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed3b91fed9f4338b4d428bd65a518cb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.778367 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.778597 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed3b91fed9f4338b4d428bd65a518cb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:34.778798 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18087874-3c01-46d8-ad06-c7e07019a259 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.786413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1337/2686] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:34 2022] DELETE /identity/v3/users/0ed3b91fed9f4338b4d428bd65a518cb => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:34.787463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.789562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.795428 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1350/2687] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:34.796427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.796718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.796997 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.817522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:34.819354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.893774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a85dd-3210-4754-9af0-3f5e35f460be admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.898994 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.906101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.906101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.906101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] PATH_INFO: `/v3/projects/efa7f2d8ca2b4bf0a1d6edbe3e32b4f6/users/21f2accc90a7447184f9c3e1a6e29a96/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.913729 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1351/2688] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:34.915110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:34.920086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa7f2d8ca2b4bf0a1d6edbe3e32b4f6, user_id=21f2accc90a7447184f9c3e1a6e29a96, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.920253 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:34.922869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:34.941451 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557c4bf-917e-4941-8ee9-361b99e621ad None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:34.946474 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b661a-36bb-4ab5-9547-781705e229ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:34.947503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b661a-36bb-4ab5-9547-781705e229ae admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:34.947828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b661a-36bb-4ab5-9547-781705e229ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:34.948081 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b661a-36bb-4ab5-9547-781705e229ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:34.949335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b661a-36bb-4ab5-9547-781705e229ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:34.951333 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b661a-36bb-4ab5-9547-781705e229ae admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.128356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1338/2689] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:34 2022] PUT /identity/v3/projects/efa7f2d8ca2b4bf0a1d6edbe3e32b4f6/users/21f2accc90a7447184f9c3e1a6e29a96/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:35.129222 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.131530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:35.195302 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1352/2690] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:34 2022] POST /identity/v3/projects => generated 369 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:35.196740 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.197001 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.197483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.215053 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5a115-4c20-4173-95f8-79bcaf0954dc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.229559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] MFA Rules not processed for user `21f2accc90a7447184f9c3e1a6e29a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:35.230854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:35.231319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:35.282903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.283593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.283877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.284157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.307382 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1353/2691] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:35.307706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.350513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.351997 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.362929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1339/2692] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:35.363778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.373801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc619e0-6097-4226-b229-d9562248b316 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.377987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3a38d-0d11-4127-bdc8-a3273ef6d60a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.381242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.382378 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.382887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.383037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.383515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.383631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.383882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.384166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.385415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.385545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.387572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.389173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.540170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1354/2693] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:35 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:35.545633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.555418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaef6f-b048-406f-9646-bf1deae705e0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.561536 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.562261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.562572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.562886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.563736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.565581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.578606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1355/2694] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:35.585881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.593453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1340/2695] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:35 2022] POST /identity/v3/projects => generated 373 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:35.609044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c620c56-e777-42ad-a9b1-467ab34c7e3a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.611956 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.657807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.676499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.676499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.676499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.676499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.676499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.676499 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1356/2696] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:35.677455 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79eb0dc-b633-4134-a404-e8ef524a38a0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.680026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.681843 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.682544 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.682820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.683094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.683830 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.685742 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.692203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f712d-90df-4e49-87ba-34aa1696889f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.697614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b7de8-0ec5-4923-b52f-ee2d90d9076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.698537 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7de8-0ec5-4923-b52f-ee2d90d9076c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.698827 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7de8-0ec5-4923-b52f-ee2d90d9076c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.699106 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b7de8-0ec5-4923-b52f-ee2d90d9076c admin admin] PATH_INFO: `/v3/projects/b3d72f29aeab4e18a7d05557dc5282ee/users/d2932fd722f4403fa276506070f97e95/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.715028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7de8-0ec5-4923-b52f-ee2d90d9076c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d72f29aeab4e18a7d05557dc5282ee, user_id=d2932fd722f4403fa276506070f97e95, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.717725 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b7de8-0ec5-4923-b52f-ee2d90d9076c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.885955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1341/2697] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:35 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:35.889047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1357/2698] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:35 2022] PUT /identity/v3/projects/b3d72f29aeab4e18a7d05557dc5282ee/users/d2932fd722f4403fa276506070f97e95/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:35.899422 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:35.902078 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.902387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.903209 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.910136 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae3400-3de5-43d9-8a6a-973dfd36c154 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:35.930944 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:35.934822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] MFA Rules not processed for user `d2932fd722f4403fa276506070f97e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:35.936141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:35.973819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:35.974719 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:35.975042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:35.975350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:35.976281 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:35.978722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:35.990078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1342/2699] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:35.995717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.001803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1358/2700] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:36.008118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb20da4-d9ed-4f0e-8ef8-a0a5e122873b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.013479 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.014631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.014631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.014903 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] PATH_INFO: `/v3/projects/b4b02a7b376a433eb14b6de7ae85331a/users/e4e460f3cba847df9ab0f5e0a959468e/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.015660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.026751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa59a6-b8db-4009-ad39-24033bcd3d66 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.031467 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b02a7b376a433eb14b6de7ae85331a, user_id=e4e460f3cba847df9ab0f5e0a959468e, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.031719 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.032392 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.032680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.032958 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.033681 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.034211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.035633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.166514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1343/2701] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:35 2022] PUT /identity/v3/projects/b4b02a7b376a433eb14b6de7ae85331a/users/e4e460f3cba847df9ab0f5e0a959468e/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:36.168234 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.170705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:36.196496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1359/2702] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:36 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:36.196496 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.253777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c337d-c778-4cb6-8adf-2db5a424b637 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.281314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3077f4-a7db-4185-ad9b-d52411b2f887 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.288508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.289226 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.289524 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.289968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.291393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.293109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.304043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1360/2703] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:36.304891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.316571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9771e3b3-c550-4b18-93c0-25d712a7e626 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.325111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.325111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.325111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.325111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.325559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.327276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.335515 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.336966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.338055 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.338476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] PATH_INFO: `/v3/projects/f0110f17b4ca48c88ff42f73c0f7e880` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.340539 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0110f17b4ca48c88ff42f73c0f7e880)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.343407 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.484245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1361/2704] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:36 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:36.485189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.496454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b08adc-f11c-4e6d-879c-1be23097966e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.501836 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.502771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.503049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.503322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] PATH_INFO: `/v3/projects/b4b02a7b376a433eb14b6de7ae85331a/users/e4e460f3cba847df9ab0f5e0a959468e/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.505442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b02a7b376a433eb14b6de7ae85331a, user_id=e4e460f3cba847df9ab0f5e0a959468e, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.508031 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.521340 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1362/2705] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:36 2022] PUT /identity/v3/projects/b4b02a7b376a433eb14b6de7ae85331a/users/e4e460f3cba847df9ab0f5e0a959468e/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:36.522227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.575992 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0110f17b4ca48c88ff42f73c0f7e880', 'request_id': 'req-a2417967-4829-403d-ba59-a1cebe549cd5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:36.589941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f1944-70a9-470f-a285-fe7419b9b72d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.657771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22694a36-b660-4e60-a811-7f5381a5d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.658461 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22694a36-b660-4e60-a811-7f5381a5d5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.658747 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22694a36-b660-4e60-a811-7f5381a5d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.659028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22694a36-b660-4e60-a811-7f5381a5d5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.659806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22694a36-b660-4e60-a811-7f5381a5d5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.661419 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22694a36-b660-4e60-a811-7f5381a5d5c5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.671733 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1363/2706] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:36.673112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.673410 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.673694 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.700439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] MFA Rules not processed for user `e4e460f3cba847df9ab0f5e0a959468e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:36.702227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:36.779121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1364/2707] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:36.780344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.795349 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253bfe0-f4f7-47b5-9046-d0be4ace3b0d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.800674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.801599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.801887 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.802164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] PATH_INFO: `/v3/projects/04e9874b49a24a1aa7a5d6d1893a043d/users/40ea5e32dc5848f6a6a07f0e8914e07b/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.819330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e9874b49a24a1aa7a5d6d1893a043d, user_id=40ea5e32dc5848f6a6a07f0e8914e07b, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:36.822823 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:36.865195 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0110f17b4ca48c88ff42f73c0f7e880', 'request_id': 'req-a2417967-4829-403d-ba59-a1cebe549cd5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:36.875638 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0110f17b4ca48c88ff42f73c0f7e880', 'request_id': 'req-a2417967-4829-403d-ba59-a1cebe549cd5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:36.892761 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] The token cache is being invalidate because project f0110f17b4ca48c88ff42f73c0f7e880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:36.893228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:36.894261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:36.896715 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1344/2708] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:36 2022] DELETE /identity/v3/projects/f0110f17b4ca48c88ff42f73c0f7e880 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:36.907535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:36.907535 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:36.971774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2417967-4829-403d-ba59-a1cebe549cd5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:36.975066 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:36.980610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:36.983065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:36.983065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:36.983065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:36.984179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:37.020303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1365/2709] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:36 2022] PUT /identity/v3/projects/04e9874b49a24a1aa7a5d6d1893a043d/users/40ea5e32dc5848f6a6a07f0e8914e07b/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:37.021853 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.024151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:37.039318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.093885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.096635 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.109752 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1345/2710] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:37.143079 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a77d5e-e230-4011-92b9-f407c648f804 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.151346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6bad5-09db-4f4b-968b-560b1b18261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.152507 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6bad5-09db-4f4b-968b-560b1b18261b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.153013 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6bad5-09db-4f4b-968b-560b1b18261b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.153505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6bad5-09db-4f4b-968b-560b1b18261b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.154741 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6bad5-09db-4f4b-968b-560b1b18261b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.157441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6bad5-09db-4f4b-968b-560b1b18261b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.175418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1366/2711] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:37.179583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.199308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59602e-f14f-4155-bfa3-004d9807c7e4 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.207148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.208627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.209093 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.209561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] PATH_INFO: `/v3/projects/04e9874b49a24a1aa7a5d6d1893a043d/users/40ea5e32dc5848f6a6a07f0e8914e07b/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.213847 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e9874b49a24a1aa7a5d6d1893a043d, user_id=40ea5e32dc5848f6a6a07f0e8914e07b, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.218930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.223774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.233939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1346/2712] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:37 2022] PUT /identity/v3/projects/04e9874b49a24a1aa7a5d6d1893a043d/users/40ea5e32dc5848f6a6a07f0e8914e07b/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:37.247048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.247966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.248683 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.251909 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:37.295977 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] MFA Rules not processed for user `40ea5e32dc5848f6a6a07f0e8914e07b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:37.297368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:37.386479 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1367/2713] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:37 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:37.402120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:37.404421 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:37.485335 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df44d94-a1cb-4400-84ce-2673e87fb6e4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:37.489079 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:37.489786 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:37.490037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:37.490312 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:37.501404 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:37.502845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:37.513633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1347/2714] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:37 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:37.745085 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:40.969309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.086830 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74829d28-230b-4cc1-9a88-693c93041808 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.092631 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.093583 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.093910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.094177 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] PATH_INFO: `/v3/users/d2932fd722f4403fa276506070f97e95` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.095412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2932fd722f4403fa276506070f97e95)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.097503 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.381098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2932fd722f4403fa276506070f97e95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.381885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2932fd722f4403fa276506070f97e95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.438530 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.440086 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:41.457854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2932fd722f4403fa276506070f97e95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.467322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2932fd722f4403fa276506070f97e95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.476192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.476833 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.481090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ac0e5786-ad2c-46f4-90d3-f0349500e3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2932fd722f4403fa276506070f97e95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.487539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1368/2715] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:40 2022] DELETE /identity/v3/users/d2932fd722f4403fa276506070f97e95 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:41.492505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.492796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.493076 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.518518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036aa0f-11f8-4f3d-9e77-a709ed18fae4 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.521992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:41.523255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:41.581962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.582892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.583205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.583683 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] PATH_INFO: `/v3/users/21f2accc90a7447184f9c3e1a6e29a96` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.585045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f2accc90a7447184f9c3e1a6e29a96)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.585990 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1369/2716] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:41.587026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.596872 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.598315 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:41.695027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995e59b-e8b2-4e4a-9682-4d3c4348fc2a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:41.699829 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60609e3-3ec6-4b6e-8573-7475c9ac6e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:41.700483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60609e3-3ec6-4b6e-8573-7475c9ac6e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.700773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60609e3-3ec6-4b6e-8573-7475c9ac6e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.701082 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60609e3-3ec6-4b6e-8573-7475c9ac6e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.710691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60609e3-3ec6-4b6e-8573-7475c9ac6e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:41.712172 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60609e3-3ec6-4b6e-8573-7475c9ac6e88 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:41.722151 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1370/2717] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:18:41.748583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f2accc90a7447184f9c3e1a6e29a96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.762470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f2accc90a7447184f9c3e1a6e29a96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.770121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f2accc90a7447184f9c3e1a6e29a96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.824554 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f2accc90a7447184f9c3e1a6e29a96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.825169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f2accc90a7447184f9c3e1a6e29a96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:41.831826 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1348/2718] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:41 2022] DELETE /identity/v3/users/21f2accc90a7447184f9c3e1a6e29a96 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:41.836354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673f206-1899-4959-8469-e63543178af8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:41.836776 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673f206-1899-4959-8469-e63543178af8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:41.836934 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673f206-1899-4959-8469-e63543178af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:41.856181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b673f206-1899-4959-8469-e63543178af8 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:41.857393 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b673f206-1899-4959-8469-e63543178af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:41.913652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:41.931584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1371/2719] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:41.945531 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:41.947252 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:42.013425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd261093-d3db-41c5-b31b-03c88c2f6054 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.017245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.018051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.018349 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.018629 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.028991 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.030446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.043798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1349/2720] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:42.145980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673f206-1899-4959-8469-e63543178af8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.147627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b673f206-1899-4959-8469-e63543178af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:42.166190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673f206-1899-4959-8469-e63543178af8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.176161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.176298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.176611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.176893 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] PATH_INFO: `/v3/projects/b3d72f29aeab4e18a7d05557dc5282ee` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.178154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d72f29aeab4e18a7d05557dc5282ee)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.180137 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.301288 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d72f29aeab4e18a7d05557dc5282ee', 'request_id': 'req-3a9fc6cc-8d08-43f4-be22-c13affa3681b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.309144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d72f29aeab4e18a7d05557dc5282ee', 'request_id': 'req-3a9fc6cc-8d08-43f4-be22-c13affa3681b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.316555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d72f29aeab4e18a7d05557dc5282ee', 'request_id': 'req-3a9fc6cc-8d08-43f4-be22-c13affa3681b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.384751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] The token cache is being invalidate because project b3d72f29aeab4e18a7d05557dc5282ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:42.385051 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.385987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.388531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1372/2721] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:42 2022] DELETE /identity/v3/projects/b3d72f29aeab4e18a7d05557dc5282ee => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:42.392480 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.399124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.404720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5acd3-0552-4bfd-be12-f05378e7c710 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.469939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.470768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.471050 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.471338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] PATH_INFO: `/v3/users/40ea5e32dc5848f6a6a07f0e8914e07b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.472593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ea5e32dc5848f6a6a07f0e8914e07b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.474578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.552194 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.553717 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:42.567839 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9fc6cc-8d08-43f4-be22-c13affa3681b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.573472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.574584 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.579613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.579613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] PATH_INFO: `/v3/projects/efa7f2d8ca2b4bf0a1d6edbe3e32b4f6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.579613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa7f2d8ca2b4bf0a1d6edbe3e32b4f6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.579613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:42.719555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ea5e32dc5848f6a6a07f0e8914e07b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.722868 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa7f2d8ca2b4bf0a1d6edbe3e32b4f6', 'request_id': 'req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.728572 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa7f2d8ca2b4bf0a1d6edbe3e32b4f6', 'request_id': 'req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.732995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ea5e32dc5848f6a6a07f0e8914e07b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.735186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa7f2d8ca2b4bf0a1d6edbe3e32b4f6', 'request_id': 'req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.741315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ea5e32dc5848f6a6a07f0e8914e07b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.801176 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] The token cache is being invalidate because project efa7f2d8ca2b4bf0a1d6edbe3e32b4f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:42.801508 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.802339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.804058 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1373/2722] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:42 2022] DELETE /identity/v3/projects/efa7f2d8ca2b4bf0a1d6edbe3e32b4f6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:42.808421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:42.809875 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:42.846995 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ea5e32dc5848f6a6a07f0e8914e07b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.847669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ea5e32dc5848f6a6a07f0e8914e07b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:42.854466 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1350/2723] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:42 2022] DELETE /identity/v3/users/40ea5e32dc5848f6a6a07f0e8914e07b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:42.879030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603b0c7-1f8e-4ba4-acb4-e27ff28a3b4b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:42.882810 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:42.941388 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:42.942043 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:42.942335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:42.942618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] PATH_INFO: `/v3/users/e4e460f3cba847df9ab0f5e0a959468e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:42.943923 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e460f3cba847df9ab0f5e0a959468e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:42.945897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:43.270515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e460f3cba847df9ab0f5e0a959468e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.271249 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e460f3cba847df9ab0f5e0a959468e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.391318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:43.392935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:43.407562 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c0a5d-ba99-4435-b26d-bcbf29c5a655 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:43.412694 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:43.414039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:43.414331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:43.414611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] PATH_INFO: `/v3/projects/04e9874b49a24a1aa7a5d6d1893a043d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:43.415852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e9874b49a24a1aa7a5d6d1893a043d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:43.426056 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:43.470245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e460f3cba847df9ab0f5e0a959468e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.480670 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e460f3cba847df9ab0f5e0a959468e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.495762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1b115678-ddc3-4742-a124-df24dd1fbdd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e460f3cba847df9ab0f5e0a959468e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.502785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1374/2724] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:42 2022] DELETE /identity/v3/users/e4e460f3cba847df9ab0f5e0a959468e => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:43.663228 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e9874b49a24a1aa7a5d6d1893a043d', 'request_id': 'req-48fd813d-2e01-44d9-b912-19bc3f934b15'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.746283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e9874b49a24a1aa7a5d6d1893a043d', 'request_id': 'req-48fd813d-2e01-44d9-b912-19bc3f934b15'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.756805 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e9874b49a24a1aa7a5d6d1893a043d', 'request_id': 'req-48fd813d-2e01-44d9-b912-19bc3f934b15'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.770723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] The token cache is being invalidate because project 04e9874b49a24a1aa7a5d6d1893a043d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:43.770723 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.771557 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-48fd813d-2e01-44d9-b912-19bc3f934b15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:43.773333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1351/2725] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:43 2022] DELETE /identity/v3/projects/04e9874b49a24a1aa7a5d6d1893a043d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:43.779834 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:43.780125 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:43.780407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:43.824705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:43.825968 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:43.906937 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1375/2726] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:43.917509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:43.917509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:43.917509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:43.930436 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:43.937150 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:43.938381 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:43.950801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1352/2727] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:43.955448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:43.966334 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29b6e-f9ac-4ec7-aca7-7bfeb7d58c09 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:43.971494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:43.972175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:43.972457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:43.972746 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:43.973473 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:43.973840 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:43.975745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:43.981067 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:43.984423 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:43.987100 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:43.993792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1376/2728] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:44.001240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.014117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96c78b-301f-431a-8258-acd451d26096 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.014321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64c095-8c6d-4b1d-bcb9-bc0988219e2d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.020396 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.020897 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.021800 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.022126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.022239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.022338 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] PATH_INFO: `/v3/projects/b4b02a7b376a433eb14b6de7ae85331a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.022439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.022619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.023354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.024747 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b02a7b376a433eb14b6de7ae85331a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.025311 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.027398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.118843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1377/2729] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:43 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:44.123271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.161725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b02a7b376a433eb14b6de7ae85331a', 'request_id': 'req-3ec2dda7-f447-4a4f-b332-ed4d98b83690'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:44.198980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b89200-296a-4c2d-9a54-0ad642be6dec admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.257574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.258256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.258538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.258813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.259541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.262674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.287071 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b02a7b376a433eb14b6de7ae85331a', 'request_id': 'req-3ec2dda7-f447-4a4f-b332-ed4d98b83690'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:44.294893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b02a7b376a433eb14b6de7ae85331a', 'request_id': 'req-3ec2dda7-f447-4a4f-b332-ed4d98b83690'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:44.305649 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] The token cache is being invalidate because project b4b02a7b376a433eb14b6de7ae85331a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:44.305939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:44.306611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec2dda7-f447-4a4f-b332-ed4d98b83690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:44.308265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1353/2730] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:43 2022] DELETE /identity/v3/projects/b4b02a7b376a433eb14b6de7ae85331a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:44.313037 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e444f-8b87-41bf-9ce2-eb0bd79f6792 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.313326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e444f-8b87-41bf-9ce2-eb0bd79f6792 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.313755 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e444f-8b87-41bf-9ce2-eb0bd79f6792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.333475 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-669e444f-8b87-41bf-9ce2-eb0bd79f6792 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:44.334725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-669e444f-8b87-41bf-9ce2-eb0bd79f6792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:44.344824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1354/2731] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:44.353169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.359543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.359543 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.372561 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:44.373840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:44.386256 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1378/2732] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/users => generated 287 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:44.386807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1355/2733] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:44.388832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.391094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.392444 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:44.404061 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8542f3-b78f-44a0-b1c8-38c905b6cda7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.404966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a24207-6eee-4f79-b513-f5c67e83b8b7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.409114 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.409814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.409985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.410110 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.410368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.410626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.410914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.411184 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.411296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:44.411611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.411938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.413211 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.413504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.424639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1356/2734] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:44.426390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1379/2735] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:44.428915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.430544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.443123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e0666-b8e0-47ca-b95e-daf863e6d660 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.443463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf179b3-0717-42ca-8e11-165bddee9cea admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.448970 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.455454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.455454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.455953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.456070 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.456169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.456277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.456373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.456482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.456582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.456582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.456752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.464618 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1357/2736] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:44.469020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.478669 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc040f4-6a99-4ac1-becd-d66f3aeaf2c0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.483721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f587-18a2-437b-8b25-adf1131e03ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.484600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f587-18a2-437b-8b25-adf1131e03ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.495656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f587-18a2-437b-8b25-adf1131e03ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.495656 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f587-18a2-437b-8b25-adf1131e03ba admin admin] PATH_INFO: `/v3/projects/49435e0aeab041c4a77d6bd38e236000/users/ae6ed5601b634acbb62683af404a7793/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.501521 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f587-18a2-437b-8b25-adf1131e03ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=49435e0aeab041c4a77d6bd38e236000, user_id=ae6ed5601b634acbb62683af404a7793, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.504251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f587-18a2-437b-8b25-adf1131e03ba admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.580709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1380/2737] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:44.584794 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.602417 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1358/2738] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:44 2022] PUT /identity/v3/projects/49435e0aeab041c4a77d6bd38e236000/users/ae6ed5601b634acbb62683af404a7793/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:44.614131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.614430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.614711 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.642612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] MFA Rules not processed for user `ae6ed5601b634acbb62683af404a7793`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:44.643906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:44.650764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77084f7-7e8c-48e6-b8cc-e8c520934b57 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.714848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.715485 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.715773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.716054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.716817 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.718691 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.719237 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1359/2739] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:44.723757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.733494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:44.738045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d5566-9349-48e8-aeb2-cc9e89dbb81d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.742891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.743590 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.743874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.744149 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.744912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.747054 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.758165 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:44.798672 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1381/2740] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:44.803224 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.814063 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bbf9f-c988-44e6-95b9-3cab1adbb6b6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.814570 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1360/2741] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:44.818509 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.875538 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.876470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.876844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.877298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.878111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.879752 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.888167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b39a37-3adf-450b-b525-5fc98d1d9a81 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.891086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1382/2742] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:44.895027 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.896245 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.896954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.897239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.897648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.898213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.900187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.906838 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae061e-ae42-439a-b0b4-9c4742ef3696 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.912707 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.913379 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.914886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.914886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.914886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.916268 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:44.927611 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1383/2743] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:44.931582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:44.941514 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2c15f-ad72-4938-aaa6-65bc66298080 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:44.946152 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c492522-5465-465b-9b1a-eb99a27931e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:44.947141 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c492522-5465-465b-9b1a-eb99a27931e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:44.947465 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c492522-5465-465b-9b1a-eb99a27931e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:44.947750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c492522-5465-465b-9b1a-eb99a27931e9 admin admin] PATH_INFO: `/v3/projects/b6f7fec7e7264be893eda5615ea759db/users/0ed082a2264e445da185e58d68b7f1fe/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:44.964261 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c492522-5465-465b-9b1a-eb99a27931e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f7fec7e7264be893eda5615ea759db, user_id=0ed082a2264e445da185e58d68b7f1fe, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:44.966969 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c492522-5465-465b-9b1a-eb99a27931e9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.024160 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1361/2744] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:44 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:45.029460 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.039731 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12368da8-50d9-4b82-9bce-0976d9363974 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.045345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.045993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.046339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.046559 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.047288 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.048905 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.049628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1384/2745] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:44 2022] PUT /identity/v3/projects/b6f7fec7e7264be893eda5615ea759db/users/0ed082a2264e445da185e58d68b7f1fe/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:45.060080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1362/2746] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:45.064087 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.064678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.066424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.066424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.075484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ac2ae-c47f-4df4-922d-c48b04e9d401 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.094602 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] MFA Rules not processed for user `0ed082a2264e445da185e58d68b7f1fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:45.095846 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:45.132197 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:45.134131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.135041 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.135317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.137230 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] PATH_INFO: `/v3/projects/05ab6e9c3a324c3781c8546f476a2c2f/users/fea6c3caddce42bf81746451032c7f57/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.153061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ab6e9c3a324c3781c8546f476a2c2f, user_id=fea6c3caddce42bf81746451032c7f57, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.155890 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.173056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1385/2747] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:45.184464 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.185936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:45.230172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1363/2748] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:45 2022] PUT /identity/v3/projects/05ab6e9c3a324c3781c8546f476a2c2f/users/fea6c3caddce42bf81746451032c7f57/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:45.233748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.248710 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a475c-bdf8-4cf3-bd62-a7f2ebe60f78 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.302581 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.303407 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.303704 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.303976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.304463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:45.315774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd98dc8-d297-4f43-a53f-ee7e5981d7bd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.322477 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.323948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.324290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.324574 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.325309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.327107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.338434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1364/2749] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:18:45.345107 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.357799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe08fb1-ac3b-40bb-94a8-9d428877bca6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.362930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbd977-a91f-4c05-acab-142c88360ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.363999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbd977-a91f-4c05-acab-142c88360ae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.364290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbd977-a91f-4c05-acab-142c88360ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.364579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbd977-a91f-4c05-acab-142c88360ae7 admin admin] PATH_INFO: `/v3/projects/05ab6e9c3a324c3781c8546f476a2c2f/users/fea6c3caddce42bf81746451032c7f57/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.366586 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbd977-a91f-4c05-acab-142c88360ae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05ab6e9c3a324c3781c8546f476a2c2f, user_id=fea6c3caddce42bf81746451032c7f57, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.369291 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbd977-a91f-4c05-acab-142c88360ae7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.372358 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.373805 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.382408 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1365/2750] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:45 2022] PUT /identity/v3/projects/05ab6e9c3a324c3781c8546f476a2c2f/users/fea6c3caddce42bf81746451032c7f57/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:18:45.383527 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1386/2751] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:18:45.394664 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.394954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.395247 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.424167 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] MFA Rules not processed for user `fea6c3caddce42bf81746451032c7f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:45.425403 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:45.503351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1366/2752] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:45.516811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:45.520614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:45.533585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ac59-a651-4306-8c3f-778291aebbc8 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:45.567957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:45.585889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4890a8-50cc-473b-8558-652c37e84b0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:45.586571 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4890a8-50cc-473b-8558-652c37e84b0d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:45.586841 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4890a8-50cc-473b-8558-652c37e84b0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:45.587123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4890a8-50cc-473b-8558-652c37e84b0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:45.647111 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4890a8-50cc-473b-8558-652c37e84b0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:45.648628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4890a8-50cc-473b-8558-652c37e84b0d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:45.658099 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1387/2753] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:48.507685 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:48.579447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33064c5d-89de-4ea3-929f-98ea912339a7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:48.584045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:48.584912 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:48.585188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:48.585462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] PATH_INFO: `/v3/users/0ed082a2264e445da185e58d68b7f1fe` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:48.586650 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed082a2264e445da185e58d68b7f1fe)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:48.589154 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:48.775643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed082a2264e445da185e58d68b7f1fe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:48.787269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:48.788514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed082a2264e445da185e58d68b7f1fe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:48.795621 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed082a2264e445da185e58d68b7f1fe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:48.910146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed082a2264e445da185e58d68b7f1fe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:48.911253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed082a2264e445da185e58d68b7f1fe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:48.918797 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1367/2754] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:18:48 2022] DELETE /identity/v3/users/0ed082a2264e445da185e58d68b7f1fe => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:48.923516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:48.923927 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:48.924120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:48.942023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:48.943248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:49.006118 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1388/2755] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:49.016537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:49.018469 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:49.103082 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337dc42-33e7-4f4c-8fd0-9e89e9a4c7b5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:49.107565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe5f80-7aa0-4afb-bc11-bb172c345f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:49.108280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe5f80-7aa0-4afb-bc11-bb172c345f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:49.108564 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe5f80-7aa0-4afb-bc11-bb172c345f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:49.108838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe5f80-7aa0-4afb-bc11-bb172c345f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:49.118550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe5f80-7aa0-4afb-bc11-bb172c345f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:49.120684 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe5f80-7aa0-4afb-bc11-bb172c345f71 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:49.132323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1368/2756] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:49.551750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:49.553229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:49.574345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fc50b-f9eb-4106-b909-35afc0d2ee5f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:49.574345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:49.574345 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:49.574709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:49.574815 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] PATH_INFO: `/v3/projects/b6f7fec7e7264be893eda5615ea759db` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:49.576101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f7fec7e7264be893eda5615ea759db)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:49.578425 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:49.665161 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f7fec7e7264be893eda5615ea759db', 'request_id': 'req-98eeaabe-9325-40cf-8243-99dffcbcad60'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:49.671284 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f7fec7e7264be893eda5615ea759db', 'request_id': 'req-98eeaabe-9325-40cf-8243-99dffcbcad60'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:49.678463 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f7fec7e7264be893eda5615ea759db', 'request_id': 'req-98eeaabe-9325-40cf-8243-99dffcbcad60'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:49.742653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] The token cache is being invalidate because project b6f7fec7e7264be893eda5615ea759db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:18:49.742945 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:49.743791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-98eeaabe-9325-40cf-8243-99dffcbcad60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:18:49.745931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1389/2757] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:18:49 2022] DELETE /identity/v3/projects/b6f7fec7e7264be893eda5615ea759db => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:49.751088 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:49.751604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:49.751705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:49.769320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:49.770528 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:49.830999 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1369/2758] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:18:49.838816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:49.839245 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:49.839862 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:49.859644 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:49.860867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:49.868871 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1390/2759] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:49.873659 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:49.883333 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d272ff-2094-4de6-92c9-e08f757ad526 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:49.889171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:49.889889 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:49.890158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:49.890514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:49.891311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:18:49.891541 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:49.893152 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:49.908471 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1370/2760] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:18:49.909854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:49.920335 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ba485-8ae4-4bb2-bcdd-3f8c89eadaba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:49.924680 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:49.925338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:49.925625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:49.925903 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:49.926604 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:49.928531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:49.931800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:50.036402 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1391/2761] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:18:49 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:18:50.040462 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:50.051358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09d28c-d0bb-4827-b5bf-d3fc36d7bc43 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:50.113734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:50.114412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.114701 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.114985 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.115728 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:50.117630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:50.190951 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:50.196506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1371/2762] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:18:50 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:18:50.201494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:50.212908 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26604144-5358-4a09-858a-7b8dcf7ebecb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:50.219715 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:50.222447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.222447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.222447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.222447 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:50.223240 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:50.237843 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1392/2763] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:50.241935 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:50.255275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9676001-9f83-487d-93af-6e8939d51582 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:50.257839 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888d41d-10b5-479d-8783-453316378872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:50.258512 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888d41d-10b5-479d-8783-453316378872 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.258801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888d41d-10b5-479d-8783-453316378872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.259108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888d41d-10b5-479d-8783-453316378872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.259877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888d41d-10b5-479d-8783-453316378872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:50.261611 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888d41d-10b5-479d-8783-453316378872 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:50.272758 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1372/2764] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:18:50.278072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:50.288494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bc057-d428-4c13-82b1-1e79065685a3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:50.294453 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:50.295964 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.296371 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.296750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] PATH_INFO: `/v3/projects/c3974a2bdfca4093986a7ca6cd926d58/users/4fca3484e66343849b9efcd87c17b1ac/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.316585 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3974a2bdfca4093986a7ca6cd926d58, user_id=4fca3484e66343849b9efcd87c17b1ac, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:50.319285 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:50.414098 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1393/2765] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:18:50 2022] PUT /identity/v3/projects/c3974a2bdfca4093986a7ca6cd926d58/users/4fca3484e66343849b9efcd87c17b1ac/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:18:50.420379 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee8a8-3d12-42e6-afba-859d0463231a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.424572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee8a8-3d12-42e6-afba-859d0463231a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.424572 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee8a8-3d12-42e6-afba-859d0463231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.440049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-847ee8a8-3d12-42e6-afba-859d0463231a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:50.441267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-847ee8a8-3d12-42e6-afba-859d0463231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:50.508487 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1373/2766] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:50.518806 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:50.520739 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:50.579787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6805984-5553-46a3-af8a-2054582a6e12 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:50.583456 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:50.584101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:50.584390 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:50.584665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:50.595144 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:50.596663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:50.606780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1394/2767] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:51.956792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:51.957357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:51.957357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:51.988157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] MFA Rules not processed for user `4fca3484e66343849b9efcd87c17b1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:51.989634 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:52.063854 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1374/2768] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:52.076223 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:52.077738 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:52.139701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53731c74-bc3f-4297-b9d9-2a2350ec8e47 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:52.143445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8bfc6-cdf2-470a-95b0-2536a7347459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:52.144101 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8bfc6-cdf2-470a-95b0-2536a7347459 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:52.144387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8bfc6-cdf2-470a-95b0-2536a7347459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:52.144666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8bfc6-cdf2-470a-95b0-2536a7347459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:52.154234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8bfc6-cdf2-470a-95b0-2536a7347459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:52.155732 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8bfc6-cdf2-470a-95b0-2536a7347459 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:52.165047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1395/2769] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:54.230885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:54.232427 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:54.246884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d792b-6424-41f8-8fe3-8f52b2caa7cb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:54.251690 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:54.252355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:54.252643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:54.252945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:54.254877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:54.323925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:54.325148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:54.342171 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1375/2770] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:55.012736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:55.013259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:55.013384 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:55.031924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] MFA Rules not processed for user `4fca3484e66343849b9efcd87c17b1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:18:55.033044 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:55.040241 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1396/2771] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:18:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:18:55.283578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:55.284910 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:55.301027 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec677f08-bf5e-42ef-bacc-1cecaaa6b708 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:55.305573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:55.306225 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:55.306513 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:55.306794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:55.309206 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:55.320419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:55.321893 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:55.326739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:55.331822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1376/2772] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:18:55.747659 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:18:55.749074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:55.814310 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:18:55.834897 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09deac2a-5074-489e-b806-41ae2cf085ee None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:18:55.840960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:18:55.841627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:18:55.841910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:18:55.842190 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:18:55.844084 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:18:55.858980 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:18:55.860660 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:18:55.871149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1397/2773] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:18:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:05.880749 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:15.746962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:15.748876 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:15.835672 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd68d-ed9b-4b57-9c52-875b6189ecef service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:15.841525 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:15.842217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:15.842504 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:15.842785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:15.844663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:15.859029 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:15.860725 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221ea9-dd8a-459f-afc0-0637c79bff77 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:15.872551 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1377/2774] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:25.867329 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=37 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:28.205625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:28.207117 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:28.220627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fd56a-178a-4d88-93b5-93098b384588 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:28.225238 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:28.226103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:28.226387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:28.226663 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] PATH_INFO: `/v3/users/ae6ed5601b634acbb62683af404a7793` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:28.227876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6ed5601b634acbb62683af404a7793)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:28.229797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:28.518347 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6ed5601b634acbb62683af404a7793'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:28.519023 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6ed5601b634acbb62683af404a7793'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:28.646898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6ed5601b634acbb62683af404a7793'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:28.654554 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6ed5601b634acbb62683af404a7793'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:28.666814 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6ed5601b634acbb62683af404a7793'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:28.673505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1398/2775] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:28 2022] DELETE /identity/v3/users/ae6ed5601b634acbb62683af404a7793 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:28.678202 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:28.679181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:28.679181 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:28.697765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:28.699311 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:28.761820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1378/2776] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:19:28.775623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:28.777303 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:28.838056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5c369-b465-4a18-8148-4f41b41bda8c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:28.844306 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:28.844994 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:28.845277 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:28.845556 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:28.855472 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:28.856951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:28.867208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1399/2777] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:29.363777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:29.365794 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:29.378207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82b167-f609-459a-a9c6-89ccbde073b3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:29.382825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:29.383930 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:29.384215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:29.384490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] PATH_INFO: `/v3/projects/49435e0aeab041c4a77d6bd38e236000` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:29.385699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] RBAC: Authorizing `identity:delete_project(project_id=49435e0aeab041c4a77d6bd38e236000)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:29.387720 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:29.591789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49435e0aeab041c4a77d6bd38e236000', 'request_id': 'req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:29.715358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49435e0aeab041c4a77d6bd38e236000', 'request_id': 'req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:29.722593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49435e0aeab041c4a77d6bd38e236000', 'request_id': 'req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:29.733194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] The token cache is being invalidate because project 49435e0aeab041c4a77d6bd38e236000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:29.738138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:29.738138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:29.738138 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1379/2778] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:29 2022] DELETE /identity/v3/projects/49435e0aeab041c4a77d6bd38e236000 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:19:29.740154 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:29.750892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91835f1f-0b5d-4a07-917a-698aad6dbcd8 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:29.829766 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:29.830764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:29.834826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:29.834826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] PATH_INFO: `/v3/users/fea6c3caddce42bf81746451032c7f57` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:29.834826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea6c3caddce42bf81746451032c7f57)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:29.834826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:29.851898 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:30.238624 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea6c3caddce42bf81746451032c7f57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:30.239458 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea6c3caddce42bf81746451032c7f57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:30.463936 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea6c3caddce42bf81746451032c7f57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:30.472421 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea6c3caddce42bf81746451032c7f57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:30.485378 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7c869a90-ca4b-45b8-a1d4-0feddc8355a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea6c3caddce42bf81746451032c7f57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:30.491771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1400/2779] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:29 2022] DELETE /identity/v3/users/fea6c3caddce42bf81746451032c7f57 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:31.009456 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:31.011355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:31.030155 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebe2cd-3441-4b8e-bcb2-f4415c05704b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:31.096724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:31.099162 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.099489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.099779 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] PATH_INFO: `/v3/projects/05ab6e9c3a324c3781c8546f476a2c2f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.101035 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05ab6e9c3a324c3781c8546f476a2c2f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:31.103002 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:31.246446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:31.267242 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05ab6e9c3a324c3781c8546f476a2c2f', 'request_id': 'req-e40e009b-5413-466c-8e28-b5baad94734e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:31.367315 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05ab6e9c3a324c3781c8546f476a2c2f', 'request_id': 'req-e40e009b-5413-466c-8e28-b5baad94734e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:31.376345 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05ab6e9c3a324c3781c8546f476a2c2f', 'request_id': 'req-e40e009b-5413-466c-8e28-b5baad94734e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:31.390637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] The token cache is being invalidate because project 05ab6e9c3a324c3781c8546f476a2c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:31.390919 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:31.391778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e40e009b-5413-466c-8e28-b5baad94734e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:31.393373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1380/2780] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:31 2022] DELETE /identity/v3/projects/05ab6e9c3a324c3781c8546f476a2c2f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:31.398600 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.399128 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.399218 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.429854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:31.431075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:31.494368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1401/2781] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:31.503514 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.503726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.503966 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.526824 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:31.528173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:31.537373 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1381/2782] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:31.543072 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:31.554056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c11e27-0d13-42aa-8daa-eea0f70e6b6a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:31.560888 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:31.561996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.563120 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.563606 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.564771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:19:31.565262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:31.567948 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:31.586331 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1402/2783] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:19:31.591072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:31.600938 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0de7e-162f-4087-98da-0b088c9ad1e4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:31.605682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:31.606339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.606625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.606906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.607633 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:31.610231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:31.701470 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1382/2784] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:19:31 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:31.705925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:31.716635 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138b1f-0ae9-4e7c-b848-24f81cff2643 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:31.775907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:31.776525 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.776811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.777119 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.777791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:31.779736 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:31.911448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1403/2785] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:19:31 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:31.915812 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:31.927326 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dbbbe-b9a7-4370-af0e-713cf54a8b64 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:31.932211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:31.932885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.933170 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.933457 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.934216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:31.935877 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:31.948594 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1383/2786] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:31.952910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:31.959859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:31.966075 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69379fc1-0bd7-454b-b932-44fa03453659 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:31.979459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad3c82-51f1-4f86-b4fb-0f6afd340fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:31.979459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3c82-51f1-4f86-b4fb-0f6afd340fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:31.979459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3c82-51f1-4f86-b4fb-0f6afd340fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:31.979459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3c82-51f1-4f86-b4fb-0f6afd340fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:31.980056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad3c82-51f1-4f86-b4fb-0f6afd340fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:31.980056 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad3c82-51f1-4f86-b4fb-0f6afd340fb2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:31.988492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1404/2787] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:31.992852 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:32.004362 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a8248-a0ab-43c4-aab5-65d3fce4172f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:32.009478 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:32.010433 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:32.010644 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:32.010921 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] PATH_INFO: `/v3/projects/d1280ccf6ac84f279f45e08e421895ba/users/c2598b1149ff4ae5b785ac6d67287422/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:32.028176 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1280ccf6ac84f279f45e08e421895ba, user_id=c2598b1149ff4ae5b785ac6d67287422, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:32.031434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:32.134891 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1384/2788] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:31 2022] PUT /identity/v3/projects/d1280ccf6ac84f279f45e08e421895ba/users/c2598b1149ff4ae5b785ac6d67287422/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:32.148877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:32.149181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:32.149491 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:32.179863 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] MFA Rules not processed for user `c2598b1149ff4ae5b785ac6d67287422`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:32.181121 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:32.249427 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1405/2789] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:19:32.255118 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:32.330945 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835f26c-36a6-4729-9809-47772d001fa6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:32.335626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:32.336330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:32.340148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:32.340148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:32.340148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:32.340148 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:32.504803 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1385/2790] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:19:32 2022] POST /identity/v3/projects => generated 383 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:32.509055 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:32.520899 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce4a72-5f64-4314-88f0-87cd9a6d046d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:32.578535 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:32.579809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:32.580086 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:32.580368 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:32.581163 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:32.583174 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.087594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1406/2791] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:19:32 2022] POST /identity/v3/users => generated 298 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:33.091741 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:33.103010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57843286-0453-4d94-bb10-5c62c936b5f3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:33.108583 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:33.109799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.110348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.110348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.111038 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:33.112706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.125219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1386/2792] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:33.129860 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:33.142092 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9143e-38cc-41a1-a69b-0d8a1618d789 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:33.146771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:33.147687 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.147966 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.148236 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] PATH_INFO: `/v3/projects/57293423fbcc47869efb3132a644e43b/users/c0d1237ea058402b9e2720ff8c0e2c0d/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.164542 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57293423fbcc47869efb3132a644e43b, user_id=c0d1237ea058402b9e2720ff8c0e2c0d, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:33.167251 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.254366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1407/2793] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:33 2022] PUT /identity/v3/projects/57293423fbcc47869efb3132a644e43b/users/c0d1237ea058402b9e2720ff8c0e2c0d/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:33.258365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:33.268663 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3e367-15b6-4146-bc6e-fed5a2d72bcc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:33.289425 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:33.326406 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0d361-861f-4418-a95b-d07f6761b559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:33.327103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d361-861f-4418-a95b-d07f6761b559 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.327410 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d361-861f-4418-a95b-d07f6761b559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.327813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0d361-861f-4418-a95b-d07f6761b559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.328626 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d361-861f-4418-a95b-d07f6761b559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:33.330395 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0d361-861f-4418-a95b-d07f6761b559 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.340628 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1387/2794] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:33.346377 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:33.357721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161db4a-3bf1-4d38-932e-3d5c0026a546 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:33.364208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:33.365179 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.365599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.365925 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] PATH_INFO: `/v3/projects/57293423fbcc47869efb3132a644e43b/users/c0d1237ea058402b9e2720ff8c0e2c0d/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.368540 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57293423fbcc47869efb3132a644e43b, user_id=c0d1237ea058402b9e2720ff8c0e2c0d, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:33.371848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.386712 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1408/2795] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:33 2022] PUT /identity/v3/projects/57293423fbcc47869efb3132a644e43b/users/c0d1237ea058402b9e2720ff8c0e2c0d/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:33.400704 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.401173 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.401651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.439293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] MFA Rules not processed for user `c0d1237ea058402b9e2720ff8c0e2c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:33.440602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:33.523813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1388/2796] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:19:33.535734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:33.539924 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:33.583132 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:33.630145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb5c2c-6b5b-4959-bd76-2e3a3e57d4b4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:33.633910 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:33.643048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.643048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.643048 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.645773 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:33.647198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.657619 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1409/2797] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:33.773320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:33.785169 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b17c66-783a-47c1-8613-e5060f38edc8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:33.789582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133d659-dc4e-4e2c-81d2-c1b592784e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:33.790309 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133d659-dc4e-4e2c-81d2-c1b592784e2d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:33.790666 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133d659-dc4e-4e2c-81d2-c1b592784e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:33.791034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133d659-dc4e-4e2c-81d2-c1b592784e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:33.859550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133d659-dc4e-4e2c-81d2-c1b592784e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:33.861573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133d659-dc4e-4e2c-81d2-c1b592784e2d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:33.871636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1389/2798] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:38.155026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:38.227615 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646aed5-20ad-4de3-a387-f296e7aee23a service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:38.232439 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:38.233318 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:38.233603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:38.233900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] PATH_INFO: `/v3/users/c2598b1149ff4ae5b785ac6d67287422` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:38.235801 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2598b1149ff4ae5b785ac6d67287422)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:38.237721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:38.509653 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2598b1149ff4ae5b785ac6d67287422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:38.510323 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2598b1149ff4ae5b785ac6d67287422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:38.589036 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2598b1149ff4ae5b785ac6d67287422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:38.597486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2598b1149ff4ae5b785ac6d67287422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:38.611610 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2598b1149ff4ae5b785ac6d67287422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:38.618122 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1410/2799] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:38 2022] DELETE /identity/v3/users/c2598b1149ff4ae5b785ac6d67287422 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:38.622967 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:38.623612 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:38.623724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:38.643780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:38.645026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:38.692036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:38.713069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1390/2800] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:38.723832 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:38.725387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:38.770972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:38.789483 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f71447-c540-484f-81e1-4a77a1a52bba admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:38.793594 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:38.794259 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:38.794545 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:38.794826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:38.806605 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:38.810551 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:38.823091 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1411/2801] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:19:39.386205 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:39.388215 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:39.403768 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c0d57-5684-444b-b254-313f9bf29cbf None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:39.414331 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:39.415575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:39.415914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:39.417346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] PATH_INFO: `/v3/projects/d1280ccf6ac84f279f45e08e421895ba` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:39.419838 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1280ccf6ac84f279f45e08e421895ba)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:39.433166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:39.609452 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1280ccf6ac84f279f45e08e421895ba', 'request_id': 'req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:39.721351 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1280ccf6ac84f279f45e08e421895ba', 'request_id': 'req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:39.729120 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1280ccf6ac84f279f45e08e421895ba', 'request_id': 'req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:39.740801 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] The token cache is being invalidate because project d1280ccf6ac84f279f45e08e421895ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:39.741069 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:39.741727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:39.743302 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1391/2802] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:39 2022] DELETE /identity/v3/projects/d1280ccf6ac84f279f45e08e421895ba => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:39.747720 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:39.760042 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961967b9-df9d-4481-b8d2-49a2426004db service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:39.837842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:39.838946 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:39.839348 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:39.839669 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] PATH_INFO: `/v3/users/c0d1237ea058402b9e2720ff8c0e2c0d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:39.841149 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d1237ea058402b9e2720ff8c0e2c0d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:39.843494 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:40.142828 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d1237ea058402b9e2720ff8c0e2c0d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:40.143632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d1237ea058402b9e2720ff8c0e2c0d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:40.298565 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d1237ea058402b9e2720ff8c0e2c0d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:40.306695 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d1237ea058402b9e2720ff8c0e2c0d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:40.328613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d33a4b83-c115-4b46-8380-01a548c3fa9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d1237ea058402b9e2720ff8c0e2c0d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:40.336586 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1412/2803] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:39 2022] DELETE /identity/v3/users/c0d1237ea058402b9e2720ff8c0e2c0d => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:19:40.959675 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:40.961365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:40.975630 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44efa2-e06f-46de-bfc4-29bf6ff6bbaf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:41.045036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:41.051483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:41.051483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:41.051483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] PATH_INFO: `/v3/projects/57293423fbcc47869efb3132a644e43b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:41.051483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57293423fbcc47869efb3132a644e43b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:41.051483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:41.221888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57293423fbcc47869efb3132a644e43b', 'request_id': 'req-c2097504-195f-4c30-8837-e53779f45c37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:41.424122 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57293423fbcc47869efb3132a644e43b', 'request_id': 'req-c2097504-195f-4c30-8837-e53779f45c37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:41.431192 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57293423fbcc47869efb3132a644e43b', 'request_id': 'req-c2097504-195f-4c30-8837-e53779f45c37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:41.439320 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:41.445717 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] The token cache is being invalidate because project 57293423fbcc47869efb3132a644e43b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:41.446040 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:41.446730 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c2097504-195f-4c30-8837-e53779f45c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:41.448348 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1392/2804] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:40 2022] DELETE /identity/v3/projects/57293423fbcc47869efb3132a644e43b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:41.453756 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:41.454513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:41.454662 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:41.474307 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:41.475482 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:41.532825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:41.541992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1413/2805] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:41.551123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:41.551459 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:41.551743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:41.571724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:41.572962 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:41.581726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1393/2806] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:41.586398 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:41.597201 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4bdd0-f194-4340-905d-42e24b8d1d81 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:41.602696 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:41.603608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:41.603760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:41.604021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:41.604809 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:19:41.605115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:41.606743 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:41.620800 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1414/2807] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:19:41.625724 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:41.636355 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661aae25-3971-446a-bbd8-c10fa3d28cb7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:41.641434 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:41.642213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:41.642430 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:41.643426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:41.643426 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:41.645365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:41.861819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1394/2808] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:19:41 2022] POST /identity/v3/projects => generated 381 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:41.866415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:41.877278 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3faaec-f8cd-4201-a321-d49b79eb5736 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:41.937166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:41.938165 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:41.940596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:41.940596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:41.940596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:41.942539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:42.180029 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1415/2809] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:19:41 2022] POST /identity/v3/users => generated 291 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:42.185001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:42.197269 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ad61-5208-493e-8945-ee2b191c43ce admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:42.209743 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:42.210874 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:42.211339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:42.211861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:42.213200 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:42.216223 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:42.236375 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1395/2810] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:42.241366 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:42.254676 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85dc69-834b-493b-8d4b-1f57b9998b71 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:42.260963 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f813a84f-3cd5-4367-be04-09ee8bd33765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:42.261664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813a84f-3cd5-4367-be04-09ee8bd33765 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:42.261957 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813a84f-3cd5-4367-be04-09ee8bd33765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:42.262239 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813a84f-3cd5-4367-be04-09ee8bd33765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:42.262983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f813a84f-3cd5-4367-be04-09ee8bd33765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:42.264652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f813a84f-3cd5-4367-be04-09ee8bd33765 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:42.277750 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1416/2811] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:42.282272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:42.293112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e6333-6d5d-4256-bf70-37d01c1bfcba admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:42.298034 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:42.298934 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:42.299219 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:42.299533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] PATH_INFO: `/v3/projects/67110b060ff34a60a9a168b506ec512f/users/c672d67ef7c14aeda6fcedb57545f0d3/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:42.318473 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=67110b060ff34a60a9a168b506ec512f, user_id=c672d67ef7c14aeda6fcedb57545f0d3, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:42.324948 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:42.603762 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1396/2812] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:42 2022] PUT /identity/v3/projects/67110b060ff34a60a9a168b506ec512f/users/c672d67ef7c14aeda6fcedb57545f0d3/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:42.616791 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:42.617523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:42.617523 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:42.646294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] MFA Rules not processed for user `c672d67ef7c14aeda6fcedb57545f0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:42.647745 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:42.724534 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1417/2813] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:42.729124 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:42.807020 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620652-062b-42da-93ba-d54d1ee92afd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:42.812680 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:42.813566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:42.813924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:42.814211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:42.814976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:42.817019 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:43.015101 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1397/2814] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:19:42 2022] POST /identity/v3/projects => generated 381 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:19:43.020264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:43.032751 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21402f-d17b-4d04-86c4-d595e8cc3c0d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:43.090140 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:43.099449 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:43.103675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:43.103675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:43.103675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:43.103675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:43.103675 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:43.301181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1418/2815] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:19:43 2022] POST /identity/v3/users => generated 297 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:43.306451 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:43.318834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e117e9e-a953-47f4-adbe-3d3f39d46d91 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:43.323631 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:43.324307 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:43.324593 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:43.324873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:43.325609 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:43.327201 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:43.339710 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1398/2816] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:43.344652 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:43.357528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113c772-2b8e-4528-a574-e412a9707c6a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:43.365135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:43.366047 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:43.366330 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:43.366608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] PATH_INFO: `/v3/projects/9f11e68e2cee49d0bafb1270224ecc10/users/73eac996bad745a88cbbe08d761d507d/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:43.384415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f11e68e2cee49d0bafb1270224ecc10, user_id=73eac996bad745a88cbbe08d761d507d, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:43.387115 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:43.728442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1419/2817] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:43 2022] PUT /identity/v3/projects/9f11e68e2cee49d0bafb1270224ecc10/users/73eac996bad745a88cbbe08d761d507d/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:43.732926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:43.743560 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e601f51-8f89-4ba3-8744-7a8814e0e8c6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:43.804682 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33403a-cbce-4c0c-9c3a-e5ae07503f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:43.805350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33403a-cbce-4c0c-9c3a-e5ae07503f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:43.805636 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33403a-cbce-4c0c-9c3a-e5ae07503f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:43.805909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33403a-cbce-4c0c-9c3a-e5ae07503f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:43.806627 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33403a-cbce-4c0c-9c3a-e5ae07503f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:43.808274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33403a-cbce-4c0c-9c3a-e5ae07503f51 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:43.818239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1399/2818] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:43.822921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:43.836825 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454bd79-c80e-479f-b46a-41b9bbfb45b5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:43.842150 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:43.843764 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:43.850596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:43.850596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] PATH_INFO: `/v3/projects/9f11e68e2cee49d0bafb1270224ecc10/users/73eac996bad745a88cbbe08d761d507d/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:43.850596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f11e68e2cee49d0bafb1270224ecc10, user_id=73eac996bad745a88cbbe08d761d507d, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:43.850596 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:43.862668 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1420/2819] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:43 2022] PUT /identity/v3/projects/9f11e68e2cee49d0bafb1270224ecc10/users/73eac996bad745a88cbbe08d761d507d/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:43.873340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:43.873686 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:43.873975 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:43.881211 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:43.901174 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] MFA Rules not processed for user `73eac996bad745a88cbbe08d761d507d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:43.902390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:43.972246 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1400/2820] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:43.983094 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:43.985460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:44.044294 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25507322-ce36-4cbc-ad56-59cf3f02a538 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:44.048889 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31571350-a906-43af-b6fd-dde8b32a815b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:44.049612 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31571350-a906-43af-b6fd-dde8b32a815b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:44.049896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31571350-a906-43af-b6fd-dde8b32a815b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:44.050173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31571350-a906-43af-b6fd-dde8b32a815b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:44.063175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31571350-a906-43af-b6fd-dde8b32a815b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:44.064671 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31571350-a906-43af-b6fd-dde8b32a815b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:44.074599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1421/2821] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:44.932398 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:45.002032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b426-0424-4f98-9aeb-80c140589acf None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:45.006820 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:45.007726 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:45.008015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:45.008297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] PATH_INFO: `/v3/users/c672d67ef7c14aeda6fcedb57545f0d3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:45.009508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c672d67ef7c14aeda6fcedb57545f0d3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:45.011449 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:45.508969 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c672d67ef7c14aeda6fcedb57545f0d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:45.522918 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c672d67ef7c14aeda6fcedb57545f0d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:45.530146 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c672d67ef7c14aeda6fcedb57545f0d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:45.675767 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c672d67ef7c14aeda6fcedb57545f0d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:45.676622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c672d67ef7c14aeda6fcedb57545f0d3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:45.683749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1401/2822] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:44 2022] DELETE /identity/v3/users/c672d67ef7c14aeda6fcedb57545f0d3 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:45.688541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:45.689147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:45.689147 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:45.709237 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:45.710492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:45.774812 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1422/2823] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:19:45.786600 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:45.788147 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:45.805458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:45.867065 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb96a23-3735-472f-942a-901cea110b67 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:45.871044 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:45.875207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:45.875207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:45.875207 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:45.882500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:45.883972 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:45.894048 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1402/2824] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:46.471638 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:46.475116 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:46.488247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd518-a794-40e6-99df-80dd13f8cf62 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:46.500417 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:46.501313 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:46.501599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:46.501878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] PATH_INFO: `/v3/projects/67110b060ff34a60a9a168b506ec512f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:46.503274 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67110b060ff34a60a9a168b506ec512f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:46.505255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:46.632648 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:46.637227 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67110b060ff34a60a9a168b506ec512f', 'request_id': 'req-03c8255f-e932-4f19-8aa9-b40e404a8d05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:46.643915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67110b060ff34a60a9a168b506ec512f', 'request_id': 'req-03c8255f-e932-4f19-8aa9-b40e404a8d05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:46.650089 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67110b060ff34a60a9a168b506ec512f', 'request_id': 'req-03c8255f-e932-4f19-8aa9-b40e404a8d05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:46.766813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] The token cache is being invalidate because project 67110b060ff34a60a9a168b506ec512f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:46.767515 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:46.767859 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:46.769467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1423/2825] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:46 2022] DELETE /identity/v3/projects/67110b060ff34a60a9a168b506ec512f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:46.773506 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:46.783791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdffdc1-6dd5-4b16-a9a0-1300a380dce5 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:46.849337 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:46.850233 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:46.850507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:46.850857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] PATH_INFO: `/v3/users/73eac996bad745a88cbbe08d761d507d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:46.852189 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73eac996bad745a88cbbe08d761d507d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:46.854140 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:47.147121 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73eac996bad745a88cbbe08d761d507d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:47.159789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73eac996bad745a88cbbe08d761d507d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:47.166661 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73eac996bad745a88cbbe08d761d507d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:47.303114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73eac996bad745a88cbbe08d761d507d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:47.303884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0e08adf3-c053-4c9e-85bc-fb6a4e2a0636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73eac996bad745a88cbbe08d761d507d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:47.310366 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1403/2826] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:46 2022] DELETE /identity/v3/users/73eac996bad745a88cbbe08d761d507d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:47.906965 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:47.908666 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:47.923753 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8255f-e932-4f19-8aa9-b40e404a8d05 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:47.992983 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:47.993878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:47.994170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:47.994450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] PATH_INFO: `/v3/projects/9f11e68e2cee49d0bafb1270224ecc10` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:47.995698 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f11e68e2cee49d0bafb1270224ecc10)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:47.997598 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:48.152186 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f11e68e2cee49d0bafb1270224ecc10', 'request_id': 'req-a6083d32-c7db-4329-ba48-8d03dc2dccd2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:48.158658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f11e68e2cee49d0bafb1270224ecc10', 'request_id': 'req-a6083d32-c7db-4329-ba48-8d03dc2dccd2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:48.165210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f11e68e2cee49d0bafb1270224ecc10', 'request_id': 'req-a6083d32-c7db-4329-ba48-8d03dc2dccd2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:48.303858 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] The token cache is being invalidate because project 9f11e68e2cee49d0bafb1270224ecc10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:48.304242 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:48.304883 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a6083d32-c7db-4329-ba48-8d03dc2dccd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:48.306445 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1424/2827] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:47 2022] DELETE /identity/v3/projects/9f11e68e2cee49d0bafb1270224ecc10 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:19:48.312458 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.312990 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.313137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.334193 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:48.335453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:48.362754 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:48.397411 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1404/2828] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:48.405123 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.411544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.411544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.426296 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:48.427549 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:48.436319 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1425/2829] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:48.441280 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:48.451026 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d3ec1-967a-492e-b796-c8b99e68c8ee None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:48.456946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:48.457602 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.457884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.458158 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.458883 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:19:48.459190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:48.460808 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:48.474490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1405/2830] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:19:48.479057 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:48.490344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f48155-04c5-4fcb-813e-bedf820c6cb5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:48.496724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:48.497429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.497742 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.498021 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.498705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:48.501424 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:48.677607 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1426/2831] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:19:48 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:48.682061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:48.692700 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bcf4c-948a-4ceb-8415-32bd7d148af7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:48.754231 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:48.754831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.755117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.755424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.756132 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:48.758036 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:48.841911 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1406/2832] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:19:48 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:48.846881 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:48.857886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee41d96-fda2-4ec2-bac4-f1f423702f72 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:48.863822 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:48.864497 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.864790 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.865069 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.865835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:48.867478 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:48.878885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1427/2833] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:48.883043 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:48.894757 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf19940-f483-43d9-b3fd-364454ec1a04 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:48.899518 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c941dd-4e9e-45b7-a27f-dbc50a9c7969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:48.900293 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c941dd-4e9e-45b7-a27f-dbc50a9c7969 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.900575 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c941dd-4e9e-45b7-a27f-dbc50a9c7969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.900848 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c941dd-4e9e-45b7-a27f-dbc50a9c7969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.901569 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c941dd-4e9e-45b7-a27f-dbc50a9c7969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:48.903166 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c941dd-4e9e-45b7-a27f-dbc50a9c7969 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:48.914640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1407/2834] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:48.919025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:48.939721 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c433d5-9358-4a02-827f-4e783e0283ad admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:48.945726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:48.946623 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:48.946911 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:48.947355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] PATH_INFO: `/v3/projects/42f3a5b253cf448f9b0990ae8ec17a17/users/58b588b0e6634e2f81c33c7876bb4c33/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:48.965271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f3a5b253cf448f9b0990ae8ec17a17, user_id=58b588b0e6634e2f81c33c7876bb4c33, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:48.968618 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:49.042354 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:49.048212 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1428/2835] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:48 2022] PUT /identity/v3/projects/42f3a5b253cf448f9b0990ae8ec17a17/users/58b588b0e6634e2f81c33c7876bb4c33/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:49.065503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:49.065799 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:49.066083 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:49.097831 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] MFA Rules not processed for user `58b588b0e6634e2f81c33c7876bb4c33`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:49.099072 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:49.173483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1408/2836] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:49.185244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:49.187601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:49.254433 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d36e1-9574-4a7f-9773-ebd853afc788 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:49.258198 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab4efc-1d7c-430e-8d1b-9659aa7b1966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:49.258855 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab4efc-1d7c-430e-8d1b-9659aa7b1966 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:49.259145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab4efc-1d7c-430e-8d1b-9659aa7b1966 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:49.259442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab4efc-1d7c-430e-8d1b-9659aa7b1966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:49.269588 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab4efc-1d7c-430e-8d1b-9659aa7b1966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:49.271053 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab4efc-1d7c-430e-8d1b-9659aa7b1966 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:49.280088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1429/2837] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:49.299765 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:49.332157 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:49.366032 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b11df3-dfca-45e7-a419-265c8daed4ad None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:49.370647 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:49.371546 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:49.371834 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:49.372114 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] PATH_INFO: `/v3/users/58b588b0e6634e2f81c33c7876bb4c33` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:49.373317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b588b0e6634e2f81c33c7876bb4c33)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:49.376060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:49.654254 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b588b0e6634e2f81c33c7876bb4c33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:49.666832 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b588b0e6634e2f81c33c7876bb4c33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:49.673916 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b588b0e6634e2f81c33c7876bb4c33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:49.766668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b588b0e6634e2f81c33c7876bb4c33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:49.767342 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b588b0e6634e2f81c33c7876bb4c33'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:49.774251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1409/2838] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:19:49 2022] DELETE /identity/v3/users/58b588b0e6634e2f81c33c7876bb4c33 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:49.778807 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:49.779182 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:49.779510 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:49.797613 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:49.798842 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:49.860837 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1430/2839] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:49.870791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:49.872278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:49.943045 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266d6aa-d09a-49a1-a238-23f5aec1fe25 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:49.946668 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5124a-f64e-4a08-b1d0-9002428486c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:49.947319 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5124a-f64e-4a08-b1d0-9002428486c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:49.947641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5124a-f64e-4a08-b1d0-9002428486c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:49.947924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5124a-f64e-4a08-b1d0-9002428486c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:49.958197 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5124a-f64e-4a08-b1d0-9002428486c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:49.960284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5124a-f64e-4a08-b1d0-9002428486c4 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:49.971321 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1410/2840] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:19:50.478342 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:50.480252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:50.497337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5bee4-1c8e-45c9-aea8-16880f21fb53 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:50.503231 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:50.504486 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:50.504869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:50.505260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] PATH_INFO: `/v3/projects/42f3a5b253cf448f9b0990ae8ec17a17` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:50.506649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f3a5b253cf448f9b0990ae8ec17a17)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:50.509516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:50.705905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f3a5b253cf448f9b0990ae8ec17a17', 'request_id': 'req-897d5c6d-b8c0-446a-9018-9f516caa2101'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:50.711582 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f3a5b253cf448f9b0990ae8ec17a17', 'request_id': 'req-897d5c6d-b8c0-446a-9018-9f516caa2101'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:50.717808 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f3a5b253cf448f9b0990ae8ec17a17', 'request_id': 'req-897d5c6d-b8c0-446a-9018-9f516caa2101'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:50.869522 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] The token cache is being invalidate because project 42f3a5b253cf448f9b0990ae8ec17a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:19:50.890835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:50.890835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-897d5c6d-b8c0-446a-9018-9f516caa2101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:19:50.890835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1431/2841] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:19:50 2022] DELETE /identity/v3/projects/42f3a5b253cf448f9b0990ae8ec17a17 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:50.894914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:50.895431 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:50.895553 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:50.914226 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:50.915484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:50.966857 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:50.986080 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1411/2842] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:50.994248 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:50.994649 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:50.994996 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.016065 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:51.017708 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:51.026562 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1432/2843] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:19:51.032143 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.042306 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517fbd5-fb18-4852-a197-c077e99755cc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.048297 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.048939 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.049818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.049818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.050256 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:19:51.050511 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.052142 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.065385 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1412/2844] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:19:51.069989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.081103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfb4e-e2eb-4da6-ad42-dc7642141011 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.086280 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.096064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.096064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.096064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.096064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.096064 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.116346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:19:51.266792 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1433/2845] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:19:51 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:51.271482 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.283052 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13518c9d-6852-4923-9ad0-47c30c5b3790 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.345667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.346235 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.346540 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.346822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.347565 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.349610 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.471216 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1413/2846] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:19:51 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:19:51.475760 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.486418 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bde54b-d6dc-433a-9a3e-b0a530d61fb5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.491555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.492387 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.493988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.493988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.493988 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.495105 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.506920 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1434/2847] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:19:51.511705 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.525496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e74138-85a0-41df-bbc9-5be865409831 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.531214 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bf8da-672b-42cb-a33c-410ecc247ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.532039 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bf8da-672b-42cb-a33c-410ecc247ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.532352 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bf8da-672b-42cb-a33c-410ecc247ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.532739 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bf8da-672b-42cb-a33c-410ecc247ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.533484 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bf8da-672b-42cb-a33c-410ecc247ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.535096 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bf8da-672b-42cb-a33c-410ecc247ca0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.548262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1414/2848] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:19:51.552462 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.564003 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39df7c1-cc2e-4b11-968b-ce11a3a2131f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.569254 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.570153 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.570431 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.570705 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] PATH_INFO: `/v3/projects/45152744cd0f46708383d5ea1d208e7a/users/b5f916c5d21f4fdba04d208730766846/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.587835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45152744cd0f46708383d5ea1d208e7a, user_id=b5f916c5d21f4fdba04d208730766846, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.590674 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.662629 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1435/2849] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:19:51 2022] PUT /identity/v3/projects/45152744cd0f46708383d5ea1d208e7a/users/b5f916c5d21f4fdba04d208730766846/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:19:51.673341 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.673749 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.674042 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.703283 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] MFA Rules not processed for user `b5f916c5d21f4fdba04d208730766846`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:19:51.704555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:51.773640 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1415/2850] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:19:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:19:51.784813 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:19:51.786558 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:19:51.852755 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29d5aa-3a50-4560-8eed-7b68298298e0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:19:51.856492 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a397d-d1c2-456a-8792-5cef049597cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:19:51.857159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a397d-d1c2-456a-8792-5cef049597cc service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:19:51.857448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a397d-d1c2-456a-8792-5cef049597cc service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:19:51.857771 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a397d-d1c2-456a-8792-5cef049597cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:19:51.868947 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a397d-d1c2-456a-8792-5cef049597cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:19:51.870442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a397d-d1c2-456a-8792-5cef049597cc service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:19:51.882568 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1436/2851] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:19:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:01.770537 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:01.876931 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:26.172463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:26.247734 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d90227-75fb-4b63-8b13-454f39f8346a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:26.253775 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:26.254678 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:26.255047 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:26.255648 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] PATH_INFO: `/v3/users/b5f916c5d21f4fdba04d208730766846` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:26.257376 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f916c5d21f4fdba04d208730766846)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:26.258924 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:26.583463 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f916c5d21f4fdba04d208730766846'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:26.596085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f916c5d21f4fdba04d208730766846'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:26.604384 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f916c5d21f4fdba04d208730766846'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:26.686323 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f916c5d21f4fdba04d208730766846'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:26.686955 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f916c5d21f4fdba04d208730766846'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:26.699123 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1416/2852] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:20:26 2022] DELETE /identity/v3/users/b5f916c5d21f4fdba04d208730766846 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:20:26.704305 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:26.705271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:26.705271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:26.722795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:26.724067 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:26.784559 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1437/2853] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:20:26.794866 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:26.796424 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:26.860018 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09889bd1-dd3d-4ff9-b55f-979d5b133539 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:26.863622 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:26.864278 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:26.864605 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:26.864892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:26.875819 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:26.877261 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:26.887550 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1417/2854] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:27.336167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:27.337921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:27.360466 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226d9d1-3ff1-41d3-980b-c7770a0f9b54 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:27.367506 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:27.368355 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:27.368642 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:27.368918 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] PATH_INFO: `/v3/projects/45152744cd0f46708383d5ea1d208e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:27.370608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=45152744cd0f46708383d5ea1d208e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:27.372826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:27.479262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45152744cd0f46708383d5ea1d208e7a', 'request_id': 'req-7b793be5-e7fc-45b0-ae2f-3d41350823bd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:27.485892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45152744cd0f46708383d5ea1d208e7a', 'request_id': 'req-7b793be5-e7fc-45b0-ae2f-3d41350823bd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:27.492450 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45152744cd0f46708383d5ea1d208e7a', 'request_id': 'req-7b793be5-e7fc-45b0-ae2f-3d41350823bd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:27.567329 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] The token cache is being invalidate because project 45152744cd0f46708383d5ea1d208e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:20:27.567627 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:27.568267 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7b793be5-e7fc-45b0-ae2f-3d41350823bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:20:27.569826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1438/2855] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:20:27 2022] DELETE /identity/v3/projects/45152744cd0f46708383d5ea1d208e7a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:20:29.057796 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:29.059253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:29.089791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:29.161051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26315e-2bc9-4684-9feb-a7b7327c7728 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:29.224279 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:29.224810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:29.225098 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:29.225380 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:29.227316 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:29.239835 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:29.241339 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:29.253759 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1418/2856] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:20:37.564608 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:39.384651 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=57 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:43.170374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0146166-d2c8-4d30-9495-2c1ba5bb89fb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:43.170973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0146166-d2c8-4d30-9495-2c1ba5bb89fb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:43.170973 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0146166-d2c8-4d30-9495-2c1ba5bb89fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:43.191164 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c0146166-d2c8-4d30-9495-2c1ba5bb89fb None None] MFA Rules not processed for user `4fca3484e66343849b9efcd87c17b1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:43.192452 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c0146166-d2c8-4d30-9495-2c1ba5bb89fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:43.257748 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1439/2857] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:43.509190 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:20:43.510900 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:43.586619 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3ee69-94fd-4126-9285-1a5d343cd499 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:20:43.590698 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:20:43.591358 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:43.591667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:43.591946 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:43.593869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:43.605060 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:20:43.606503 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:20:43.615566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1419/2858] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:20:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:20:43.745004 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:20:43.745406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:20:43.745785 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:20:43.768200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] MFA Rules not processed for user `f8cc86b384df4223a800838a3d386d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:20:43.769435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:20:43.831192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1440/2859] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:20:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:20:53.611534 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=26 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:20:53.822603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=50 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:28.670689 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:28.681008 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef358d41-20b5-4572-8b6c-ab695ccbd43c service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:28.684791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:28.685447 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:28.685756 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:28.686159 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:28.688010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:28.777896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:28.779336 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:28.802708 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1420/2860] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:29.994344 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:21:30.005035 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7dd44-bb27-44c5-99f6-4961d09307dd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:21:30.010229 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:21:30.011300 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:21:30.011811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:21:30.012276 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:21:30.015133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:21:30.037678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:21:30.041896 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:21:30.056688 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1441/2861] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:21:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:21:38.799861 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=43 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:21:40.050759 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:07.675986 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:07.677706 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:07.694015 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4859-9193-4e75-aa0a-c1515488acb9 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:07.699773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:07.700445 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:07.700773 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:07.701049 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:07.703898 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:07.717365 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:07.724784 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:07.731016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1421/2862] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:07.871905 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:07.874787 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:07.961269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3e03-c16c-4a28-85cc-0b0e970bbadc service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:07.966133 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:07.967022 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:07.967415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:07.967763 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] PATH_INFO: `/v3/users/4fca3484e66343849b9efcd87c17b1ac` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:07.969026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fca3484e66343849b9efcd87c17b1ac)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:07.971028 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:08.228166 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fca3484e66343849b9efcd87c17b1ac'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:08.228818 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fca3484e66343849b9efcd87c17b1ac'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:08.344438 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fca3484e66343849b9efcd87c17b1ac'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:08.354728 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fca3484e66343849b9efcd87c17b1ac'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:08.368429 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-15a44361-029a-4294-8dc1-8adddad12ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fca3484e66343849b9efcd87c17b1ac'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:08.374664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1442/2863] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:22:07 2022] DELETE /identity/v3/users/4fca3484e66343849b9efcd87c17b1ac => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:22:08.671654 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:08.685127 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f6470-8c3e-40c9-adaa-7cefc8dbbd2f service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:08.750495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:08.751275 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:08.751604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:08.751888 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] PATH_INFO: `/v3/projects/c3974a2bdfca4093986a7ca6cd926d58` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:08.753217 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3974a2bdfca4093986a7ca6cd926d58)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:08.755128 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:08.888687 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3974a2bdfca4093986a7ca6cd926d58', 'request_id': 'req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:08.995094 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3974a2bdfca4093986a7ca6cd926d58', 'request_id': 'req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:09.006646 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3974a2bdfca4093986a7ca6cd926d58', 'request_id': 'req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:09.021993 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] The token cache is being invalidate because project c3974a2bdfca4093986a7ca6cd926d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:22:09.022284 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:09.022953 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e32e21f0-22c8-42c0-ae50-bbc74f216a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:22:09.024555 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1422/2864] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:22:08 2022] DELETE /identity/v3/projects/c3974a2bdfca4093986a7ca6cd926d58 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:22:09.031255 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.031848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.031848 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.051068 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:09.052544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:09.117907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1443/2865] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:22:09.126813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.127957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.127957 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.146318 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:09.147578 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:09.156282 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1423/2866] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:09.165907 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:09.176874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d3746-fc6e-4179-b202-f1ce8b2768a7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:09.183273 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:09.184210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.184614 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.184992 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.186103 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:22:09.186555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:09.188658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:09.201576 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1444/2867] 10.222.0.58 () {58 vars in 1152 bytes} [Thu Jun 16 18:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:22:09.210267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:09.221925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4102c9-b451-4ea8-b214-83e75cce1a3f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:09.226617 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:09.227325 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.227681 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.227960 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.228652 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:09.230545 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:09.450785 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1424/2868] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:22:09 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:22:09.452599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:09.463269 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95d1cf-ad0c-4018-9550-dac1625f3539 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:09.468167 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:09.525435 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:09.526126 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.526412 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.526692 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.527416 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:09.529339 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:09.767885 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1445/2869] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:22:09 2022] POST /identity/v3/users => generated 292 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:22:09.772179 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:09.782981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c76f6-d1d4-433c-98dc-43fb8587af4d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:09.787931 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:09.788598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.788884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.789161 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.789887 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:09.791500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:09.805239 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1425/2870] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:22:09.809891 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:09.823087 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405439-a2df-4884-a158-b62e41fe60b7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:09.829116 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f106e2b-02c8-41f5-a236-aabc31cbd1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:09.829777 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106e2b-02c8-41f5-a236-aabc31cbd1de admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.830062 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106e2b-02c8-41f5-a236-aabc31cbd1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.830338 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106e2b-02c8-41f5-a236-aabc31cbd1de admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.831074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f106e2b-02c8-41f5-a236-aabc31cbd1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:09.832993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f106e2b-02c8-41f5-a236-aabc31cbd1de admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:09.847415 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1446/2871] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:22:09.851301 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:09.862164 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928fbdb-f61a-4cdb-99b2-af9a6cdd9001 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:09.867102 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:09.868016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:09.868305 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:09.868582 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] PATH_INFO: `/v3/projects/60fd87cc7cb24ee0a0006089d4b13084/users/e3f73a99241e449bbee5e0a28878e80c/roles/4ef493d63de24283a7dff2172a9c856c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:09.888807 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fd87cc7cb24ee0a0006089d4b13084, user_id=e3f73a99241e449bbee5e0a28878e80c, role_id=4ef493d63de24283a7dff2172a9c856c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:09.892070 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:10.029507 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1426/2872] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:22:09 2022] PUT /identity/v3/projects/60fd87cc7cb24ee0a0006089d4b13084/users/e3f73a99241e449bbee5e0a28878e80c/roles/4ef493d63de24283a7dff2172a9c856c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:22:10.035195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406355f-92d8-43c9-8c9a-1374418c6350 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:10.038797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406355f-92d8-43c9-8c9a-1374418c6350 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:10.038797 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406355f-92d8-43c9-8c9a-1374418c6350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:10.055929 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8406355f-92d8-43c9-8c9a-1374418c6350 None None] MFA Rules not processed for user `d088ec4f4ba04369a9931bed85bdd327`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:10.057210 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8406355f-92d8-43c9-8c9a-1374418c6350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:10.129941 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1447/2873] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:10.140811 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:10.142587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:10.159490 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:10.210209 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47cdf-e05b-481b-8ac2-dd71157ee057 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:10.213129 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:10.213790 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:10.214077 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:10.214357 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:10.225343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:10.226777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:10.245187 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1427/2874] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:17.449025 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:17.449780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:17.449780 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:17.498628 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] MFA Rules not processed for user `e3f73a99241e449bbee5e0a28878e80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:17.500090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:17.607900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1448/2875] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:22:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:17.612566 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:17.624954 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47abfb9-5014-4fa8-a576-4699cb0dfe52 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:17.630308 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:17.631051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:17.631343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:17.631658 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:17.632390 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:17.634579 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:17.801771 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1428/2876] 10.222.0.58 () {60 vars in 1152 bytes} [Thu Jun 16 18:22:17 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:22:17.806030 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:17.832601 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e34415-87c9-47f9-ab70-775f78cb314d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:17.845337 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:17.900195 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:17.901175 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:17.901564 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:17.901877 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:17.902854 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:17.905912 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:18.012295 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1449/2877] 10.222.0.58 () {60 vars in 1143 bytes} [Thu Jun 16 18:22:17 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:22:18.016722 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:18.034346 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392149c-7d3c-4396-919f-65e3595345d7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:18.040927 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:18.041813 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:18.042265 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:18.042842 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:18.043809 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:18.046929 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:18.061789 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1429/2878] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:22:18.066083 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:18.078574 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb890e-bf4d-410e-9629-3fe031cecd9d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:18.084555 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:18.085507 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:18.085795 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:18.086074 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] PATH_INFO: `/v3/projects/5252b7098518458ba8caefe1a7dc234b/users/93f413ea48af42e4b8a527d36c7fc8d5/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:18.106187 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5252b7098518458ba8caefe1a7dc234b, user_id=93f413ea48af42e4b8a527d36c7fc8d5, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:18.109951 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:18.293208 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1450/2879] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:22:18 2022] PUT /identity/v3/projects/5252b7098518458ba8caefe1a7dc234b/users/93f413ea48af42e4b8a527d36c7fc8d5/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:22:18.298791 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:18.310858 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6b882-f95e-4013-ba17-7b3cf2726a19 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:18.395573 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6f20c-92a6-4fa5-acec-4e4d69c87ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:18.396259 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6f20c-92a6-4fa5-acec-4e4d69c87ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:18.396536 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6f20c-92a6-4fa5-acec-4e4d69c87ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:18.396914 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6f20c-92a6-4fa5-acec-4e4d69c87ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:18.397643 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6f20c-92a6-4fa5-acec-4e4d69c87ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:18.399563 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6f20c-92a6-4fa5-acec-4e4d69c87ec8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:18.412862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1430/2880] 10.222.0.58 () {58 vars in 1121 bytes} [Thu Jun 16 18:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:22:18.417762 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:18.433234 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12471b78-d088-43f9-bd9a-52b29951310a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:18.442112 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:18.443386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:18.443709 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:18.444006 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] PATH_INFO: `/v3/projects/5252b7098518458ba8caefe1a7dc234b/users/93f413ea48af42e4b8a527d36c7fc8d5/roles/d1f5f979450b4d00b3d1a2189096884d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:18.446437 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5252b7098518458ba8caefe1a7dc234b, user_id=93f413ea48af42e4b8a527d36c7fc8d5, role_id=d1f5f979450b4d00b3d1a2189096884d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:18.451603 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:18.465678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1451/2881] 10.222.0.58 () {60 vars in 1482 bytes} [Thu Jun 16 18:22:18 2022] PUT /identity/v3/projects/5252b7098518458ba8caefe1a7dc234b/users/93f413ea48af42e4b8a527d36c7fc8d5/roles/d1f5f979450b4d00b3d1a2189096884d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:22:25.499340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed9b0c-9328-4040-80aa-0487bd240366 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:25.499867 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed9b0c-9328-4040-80aa-0487bd240366 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:25.500243 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed9b0c-9328-4040-80aa-0487bd240366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:25.539718 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-deed9b0c-9328-4040-80aa-0487bd240366 None None] MFA Rules not processed for user `93f413ea48af42e4b8a527d36c7fc8d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:25.541108 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-deed9b0c-9328-4040-80aa-0487bd240366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:25.644347 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1431/2882] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:22:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:25.657441 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:25.659327 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:25.749960 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36176126-f605-4a27-ad31-517fb03c32f4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:25.753811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f37017740874fef845f23fca4f977f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:25.754405 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:25.754689 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:25.754961 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:25.819389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:25.819389 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:25.830892 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1452/2883] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:26.178126 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:26.178413 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:26.178691 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:26.183777 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:26.241075 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:26.242508 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:26.312976 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1432/2884] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:22:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:26.564020 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:26.565498 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:26.569297 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:26.661049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22bc66-37d5-4d48-b4ed-aee46b371591 service glance] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:26.666264 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dced2fe-fc30-4025-a658-bf3807af209a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:26.666915 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced2fe-fc30-4025-a658-bf3807af209a service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:26.667203 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced2fe-fc30-4025-a658-bf3807af209a service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:26.667516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dced2fe-fc30-4025-a658-bf3807af209a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:26.676406 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dced2fe-fc30-4025-a658-bf3807af209a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:26.677930 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dced2fe-fc30-4025-a658-bf3807af209a service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:26.686894 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1453/2885] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:22:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:22:36.310014 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=42 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:36.688346 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=40 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:37.541671 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:37.543442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:37.608327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e3c70-03e8-4ca1-964e-b3826d4f4c60 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:37.611880 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:37.612527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:37.612810 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:37.613085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:37.614873 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:37.625814 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:37.627274 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f83af3-fed3-44cf-9c7f-985b2e2fa75d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:37.635697 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1433/2886] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:38.623685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:38.624262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:38.624262 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:38.643054 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] MFA Rules not processed for user `e3f73a99241e449bbee5e0a28878e80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:38.644459 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:38.651737 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1454/2887] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:22:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:44.754273 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:44.754776 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:44.754963 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:44.779825 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:22:44.781377 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:44.798415 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1434/2888] 10.222.0.58 () {58 vars in 1009 bytes} [Thu Jun 16 18:22:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:22:44.812563 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:44.828217 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3712f-fb0f-4038-b02e-fbb8691cdf62 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:44.833878 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:44.834633 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:44.834921 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:44.835200 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:44.845882 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:44.847658 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:44.863772 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1455/2889] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:22:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:22:46.180240 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:22:46.182981 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:46.357117 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff9fa2-bfd9-43f1-ad38-fe0b0482148a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:22:46.363958 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:22:46.364764 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:22:46.365115 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:22:46.365494 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:22:46.367625 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:22:46.385213 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:22:46.387178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b840c3-8770-4be9-bc6e-20b9fbdf2ccb service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:22:46.401137 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1435/2890] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:22:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:22:54.853213 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=9 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:22:56.396343 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=63 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:23:09.255404 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:23:09.257233 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:09.338731 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e45002-b902-4984-bfb5-6210f9d0ddd4 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:23:09.343505 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbc41720695b45ec84c2f250238ac37b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:23:09.344159 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:09.344440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:09.344718 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:09.346758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:09.425993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:23:09.427470 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:23:09.438758 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1456/2891] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:23:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:23:19.432803 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=48 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:23:21.560850 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:21.561419 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:21.561840 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:21.585932 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] MFA Rules not processed for user `f8cc86b384df4223a800838a3d386d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:23:21.587178 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:21.595059 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1436/2892] 10.222.0.58 () {58 vars in 957 bytes} [Thu Jun 16 18:23:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:23:23.191090 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:23:23.210135 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11d6d5-1497-4e99-86ca-7153df6fa779 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:23:23.215501 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:23:23.219173 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:23.219826 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:23.220386 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:23.242185 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:23:23.244460 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:23:23.262109 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1457/2893] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:23:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:23:29.129500 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:23:29.131006 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:29.161782 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a9ec8-df19-4eb3-9415-c532adb8070d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:23:29.189667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9279195b1ed64901a63a3fce8d0bb610', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:23:29.189667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:29.189667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:29.189667 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:29.190089 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:29.191925 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:23:29.193604 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:23:29.212051 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1437/2894] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:23:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:23:33.252811 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:23:39.203798 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:23:44.987359 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:23:44.999544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c73f-5f1b-4453-957e-9a91a4f62471 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:23:45.005900 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:23:45.006972 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:23:45.007665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:23:45.008170 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:23:45.011724 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:23:45.107678 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:23:45.109955 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fc45a-6352-42af-acf6-b2170ba8f47c service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:23:45.130189 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1458/2895] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:23:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:23:55.120244 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=30 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:25:35.000845 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:35.002327 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:35.073778 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7971-77dc-49ec-b7c2-977dec5e2ce7 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:35.078853 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:35.079533 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:35.079822 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:35.080103 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:35.081915 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:35.094106 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:35.095727 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:35.106389 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1438/2896] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:25:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:25:42.237181 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:42.237685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:42.237796 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:42.256341 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] MFA Rules not processed for user `df08ee888dfd4aeb8025b594fcb2702e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:25:42.257569 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:42.265553 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1459/2897] 10.222.0.58 () {58 vars in 967 bytes} [Thu Jun 16 18:25:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:25:42.308131 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:42.319495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7f828-0b22-41c1-863b-beaf45e9c663 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:42.325350 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd57a7e4c074ca2890ec6bc081809cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:42.325987 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:42.326272 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:42.326547 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:42.335446 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:42.337356 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:42.348892 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1439/2898] 10.222.0.58 () {60 vars in 1384 bytes} [Thu Jun 16 18:25:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:25:52.251088 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:25:52.342262 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=38 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:25:54.074528 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:54.144734 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ca7c8-d91e-4928-9398-0f19b61d0b33 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:54.150374 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:54.151289 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:54.151632 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:54.151959 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] PATH_INFO: `/v3/users/e3f73a99241e449bbee5e0a28878e80c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:54.153183 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f73a99241e449bbee5e0a28878e80c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:54.155599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:54.635383 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f73a99241e449bbee5e0a28878e80c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:54.636077 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f73a99241e449bbee5e0a28878e80c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:54.736701 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f73a99241e449bbee5e0a28878e80c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:54.754440 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f73a99241e449bbee5e0a28878e80c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:54.757026 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f73a99241e449bbee5e0a28878e80c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:54.764018 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1460/2899] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:25:54 2022] DELETE /identity/v3/users/e3f73a99241e449bbee5e0a28878e80c => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:25:55.103188 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:55.114527 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13657-1d1c-4bc0-a25c-136b70df13c2 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:55.180862 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:55.181941 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:55.182268 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:55.182496 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] PATH_INFO: `/v3/projects/60fd87cc7cb24ee0a0006089d4b13084` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:55.183926 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=60fd87cc7cb24ee0a0006089d4b13084)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:55.185896 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:55.840909 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60fd87cc7cb24ee0a0006089d4b13084', 'request_id': 'req-6f998790-b977-4627-8522-b9203cebd1de'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:55.905192 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:55.907322 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:55.982625 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac4032-f349-49a5-b918-cfddea62f2f1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:56.016457 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:25:56.033587 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6aae965af8284363820254c9286adf96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:56.034247 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:56.034541 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:56.034816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:56.036748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60fd87cc7cb24ee0a0006089d4b13084', 'request_id': 'req-6f998790-b977-4627-8522-b9203cebd1de'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.044516 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:56.044699 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60fd87cc7cb24ee0a0006089d4b13084', 'request_id': 'req-6f998790-b977-4627-8522-b9203cebd1de'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.062637 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] The token cache is being invalidate because project 60fd87cc7cb24ee0a0006089d4b13084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:25:56.062933 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.063748 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.069253 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1440/2900] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:25:55 2022] DELETE /identity/v3/projects/60fd87cc7cb24ee0a0006089d4b13084 => generated 0 bytes in 964 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:25:56.070317 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:56.073267 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:56.110260 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:56.111886 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:56.123664 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1461/2901] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:25:56.158025 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f998790-b977-4627-8522-b9203cebd1de admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:56.165869 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:56.167489 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:56.168010 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:56.168529 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] PATH_INFO: `/v3/users/93f413ea48af42e4b8a527d36c7fc8d5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:56.170906 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f413ea48af42e4b8a527d36c7fc8d5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:56.174780 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:56.519746 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:25:56.537884 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f413ea48af42e4b8a527d36c7fc8d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.560442 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f413ea48af42e4b8a527d36c7fc8d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.573885 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f413ea48af42e4b8a527d36c7fc8d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.666829 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f413ea48af42e4b8a527d36c7fc8d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.668290 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f413ea48af42e4b8a527d36c7fc8d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:56.677964 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1441/2902] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:25:56 2022] DELETE /identity/v3/users/93f413ea48af42e4b8a527d36c7fc8d5 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:25:57.054685 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:25:57.055835 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:25:57.079609 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e77b36-cc90-496a-badc-5d973564c615 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:25:57.173049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:25:57.173049 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:25:57.174271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:25:57.174271 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] PATH_INFO: `/v3/projects/5252b7098518458ba8caefe1a7dc234b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:25:57.175448 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5252b7098518458ba8caefe1a7dc234b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:25:57.177467 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:25:57.326956 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5252b7098518458ba8caefe1a7dc234b', 'request_id': 'req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:57.340513 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5252b7098518458ba8caefe1a7dc234b', 'request_id': 'req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:57.350856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5252b7098518458ba8caefe1a7dc234b', 'request_id': 'req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:57.440444 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] The token cache is being invalidate because project 5252b7098518458ba8caefe1a7dc234b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:25:57.442976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:57.442976 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:25:57.446138 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1462/2903] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:25:57 2022] DELETE /identity/v3/projects/5252b7098518458ba8caefe1a7dc234b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:26:04.714085 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:04.716476 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:04.732078 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa452-ca3e-47f5-84d5-e4ce1397e36c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:04.790863 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:04.791774 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:04.792061 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:04.792340 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] PATH_INFO: `/v3/users/f8cc86b384df4223a800838a3d386d3f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:04.793542 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8cc86b384df4223a800838a3d386d3f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:04.795598 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:05.102370 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8cc86b384df4223a800838a3d386d3f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:05.114523 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8cc86b384df4223a800838a3d386d3f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:05.122412 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8cc86b384df4223a800838a3d386d3f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:05.313721 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8cc86b384df4223a800838a3d386d3f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:05.314344 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8cc86b384df4223a800838a3d386d3f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:05.321589 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1442/2904] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:26:04 2022] DELETE /identity/v3/users/f8cc86b384df4223a800838a3d386d3f => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:26:05.878145 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:05.879816 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:05.900620 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e33d9-f0f4-4c87-a789-8b89b008c4f3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:05.915778 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:05.963916 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:05.964939 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:05.965309 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:05.965599 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] PATH_INFO: `/v3/projects/bff5c01fd0ab4348b4704765f0195d50` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:05.967544 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff5c01fd0ab4348b4704765f0195d50)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:05.969539 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:06.147856 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff5c01fd0ab4348b4704765f0195d50', 'request_id': 'req-a9acc29c-e47b-4320-ace9-d36d4c2d1225'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.157844 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff5c01fd0ab4348b4704765f0195d50', 'request_id': 'req-a9acc29c-e47b-4320-ace9-d36d4c2d1225'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.165408 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff5c01fd0ab4348b4704765f0195d50', 'request_id': 'req-a9acc29c-e47b-4320-ace9-d36d4c2d1225'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.287531 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] The token cache is being invalidate because project bff5c01fd0ab4348b4704765f0195d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:26:06.287867 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.288520 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.290706 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1463/2905] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:26:05 2022] DELETE /identity/v3/projects/bff5c01fd0ab4348b4704765f0195d50 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:26:06.294792 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:06.296251 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:06.309818 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b818c-b36c-4bf7-8a89-220d490ef156 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:06.337328 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:06.367937 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:06.378642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:06.378642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:06.378642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] PATH_INFO: `/v3/users/fc608f0b89744843a231d2b6365d257d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:06.378642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc608f0b89744843a231d2b6365d257d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:06.378642 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:06.723453 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc608f0b89744843a231d2b6365d257d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.736112 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc608f0b89744843a231d2b6365d257d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.743639 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc608f0b89744843a231d2b6365d257d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.897030 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc608f0b89744843a231d2b6365d257d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.897788 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc608f0b89744843a231d2b6365d257d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:06.905587 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1443/2906] 10.222.0.58 () {58 vars in 1223 bytes} [Thu Jun 16 18:26:06 2022] DELETE /identity/v3/users/fc608f0b89744843a231d2b6365d257d => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:26:07.326864 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:26:07.328714 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:26:07.344442 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acc29c-e47b-4320-ace9-d36d4c2d1225 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:26:07.427252 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd088ec4f4ba04369a9931bed85bdd327', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24af7229251445019ca42ea240bb437e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:26:07.428298 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:26:07.428455 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:26:07.428726 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] PATH_INFO: `/v3/projects/ec8d64f5e76a4b28a4583aca81f79e97` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:26:07.430845 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8d64f5e76a4b28a4583aca81f79e97)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:26:07.432989 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:26:07.559876 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:07.561314 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8d64f5e76a4b28a4583aca81f79e97', 'request_id': 'req-9d079b7b-cb97-474e-a2cf-e8f439502e2b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:07.566987 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8d64f5e76a4b28a4583aca81f79e97', 'request_id': 'req-9d079b7b-cb97-474e-a2cf-e8f439502e2b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:07.574454 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8d64f5e76a4b28a4583aca81f79e97', 'request_id': 'req-9d079b7b-cb97-474e-a2cf-e8f439502e2b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:07.669385 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] The token cache is being invalidate because project ec8d64f5e76a4b28a4583aca81f79e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:26:07.669665 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:07.670869 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9d079b7b-cb97-474e-a2cf-e8f439502e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:26:07.671993 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1464/2907] 10.222.0.58 () {58 vars in 1232 bytes} [Thu Jun 16 18:26:07 2022] DELETE /identity/v3/projects/ec8d64f5e76a4b28a4583aca81f79e97 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:26:16.901113 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:26:17.750874 dv-cinder-837543-4-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:30:41.027194 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:30:41.028641 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:41.178588 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006d3ce-f044-4287-9ce9-ef22c2c8dcda admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:30:41.241016 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efaf16b0733b426da91c5f5ae08deaac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca560e4628b041ee8e8132b2f35b4c4c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:30:41.242001 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:30:41.242172 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:30:41.242330 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:30:41.244745 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:30:41.261483 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:30:41.263450 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bec967-a7c8-47fd-a6d1-fc3b6399b8e5 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:30:41.276492 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1444/2908] 10.222.0.58 () {60 vars in 1365 bytes} [Thu Jun 16 18:30:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:30:51.271495 dv-cinder-837543-4-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}